Text data generation method and training method of question and answer model

Abstract

The embodiment of the specification provides a text data generation method and a training method of a question and answer model, wherein the text data generation method comprises the following steps: receiving a text data generation instruction for an initial sample text set, wherein the initial sample text set comprises a plurality of initial sample texts; determining at least one text expansion type in response to the text data generation instruction, and obtaining a text expansion template corresponding to each text expansion type, wherein the text expansion type comprises a multi-document retrieval type, a few-sample learning type and an abstract type; selecting an initial sample text from the initial sample text set, adding the initial sample text to a target text expansion template, and generating a target sample text corresponding to the target text expansion template, wherein the target text expansion template is any one of the text expansion templates, and the target sample text is used for training the question and answer model.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to methods for generating text data and training methods for question-answering models. Background Technology

[0002] Large Language Models (LLMs) represent a significant breakthrough in the field of artificial intelligence in recent years, and they have been widely applied across various domains. LLMs have achieved promising results in scenarios such as content generation, question answering, and knowledge reasoning.

[0003] Large language models face certain challenges when processing long texts. This is due to the limited number of long text samples currently available for training these models, as well as the high computational demands. To enrich the long text samples available to large language models, a common approach is to fine-tune instructions to construct long texts. However, current methods for constructing long text samples typically rely on the capabilities of the large language model or require manual intervention, resulting in high costs for generating a sufficient number of long text samples. Furthermore, the diversity of the generated long text samples is limited by the generation capabilities of the large language model. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a text data generation method and a question-answering model training method. One or more embodiments of this specification also relate to a text data generation apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a text data generation method is provided, comprising: Receive a text data generation instruction for an initial sample text set, wherein the initial sample text set includes multiple initial sample texts; In response to the text data generation instruction, at least one text augmentation type is determined, and a text augmentation template corresponding to each text augmentation type is obtained, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type; Initial sample text is selected from the initial sample text set and added to the target text expansion template to generate the target sample text corresponding to the target text expansion template. The target text expansion template is any one of the text expansion templates, and the target sample text is used to train the question answering model.

[0006] According to a second aspect of the embodiments of this specification, a text data generation method is provided, applied to a cloud-side device, comprising: Text data generation instructions sent by the receiving end device; Obtain the initial sample text set corresponding to the text data generation instruction, wherein the initial sample text set includes multiple initial sample texts; In response to the text data generation instruction, at least one text augmentation type is determined, and a text augmentation template corresponding to each text augmentation type is obtained, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type; Initial sample text is selected from the initial sample text set and added to the target text expansion template to generate the target sample text corresponding to the target text expansion template. The target text expansion template is any one of the text expansion templates, and the target sample text is used to train the question answering model. Each target sample text is sent to the terminal device.

[0007] According to a third aspect of the embodiments of this specification, a method for training a question-answering model is provided, applied to a cloud-based device, comprising: Text data generation instructions sent by the receiving end device; Obtain the initial sample text set corresponding to the text data generation instruction, wherein the initial sample text set includes multiple initial sample texts; In response to the text data generation instruction, at least one text augmentation type is determined, and a text augmentation template corresponding to each text augmentation type is obtained, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type; Initial sample text is selected from the initial sample text set and added to the target text expansion template to generate the target sample text corresponding to the target text expansion template, wherein the target text expansion template is any one of the text expansion templates; A question-answering model is trained based on the target sample text, the model parameters of the question-answering model are obtained, and the model parameters are sent to the edge device.

[0008] According to a fourth aspect of the embodiments of this specification, a text data generation system is provided, including a client and a server; wherein... The client is configured to send a text data generation instruction to the server for an initial sample text set, wherein the initial sample text set includes multiple initial sample texts; The server is configured to respond to the text data generation instruction to determine at least one text augmentation type and obtain text augmentation templates corresponding to each text augmentation type, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type; select initial sample text from the initial sample text set and add it to the target text augmentation template to generate target sample text corresponding to the target text augmentation template, wherein the target text augmentation template is any one of the text augmentation templates, and the target sample text is used to train the question answering model; and send the target sample text to the client.

[0009] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above method.

[0010] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0011] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0012] The method provided in this specification, based on an initial sample text set, aims to generate target sample text by producing long text instruction fine-tuning data for training question-answering models to enhance their multi-document retrieval, few-shot learning, and summarization capabilities. During the generation process, different text augmentation types are determined based on different capabilities, and text augmentation templates corresponding to these types are obtained. Initial sample texts from the initial sample text set are randomly added to the text augmentation templates to generate the corresponding target sample texts. Utilizing the initial sample text set to generate different types of long text instruction fine-tuning data enriches the diversity of long text instruction fine-tuning data, eliminates the need for large language models or manual intervention, saves data acquisition costs, and the method of concatenating the initial sample texts makes the length of the final generated target sample text controllable. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating a text data generation method provided in one embodiment of this specification; Figure 2 This is a flowchart illustrating the processing steps of a long text instruction fine-tuning data generation method provided in one embodiment of this specification. Figure 3 This is a schematic diagram of the structure of a text data generation device provided in one embodiment of this specification; Figure 4 This is a flowchart of a text data generation method for cloud-side devices provided in one embodiment of this specification; Figure 5 This is a flowchart of a training method for a question-answering model applied to a cloud-based device, provided in one embodiment of this specification. Figure 6 This is an architecture diagram of a text data generation system provided in one embodiment of this specification; Figure 7 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0014] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0015] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0016] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0017] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0018] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0019] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0020] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0021] Supervised Fine-Tuning Data (SFT) refers to datasets used to refine (fine-tune) machine learning models in a supervised learning environment. This type of data is typically labeled, meaning each data sample has a clear label or output, enabling the model to learn the relationship between inputs and outputs. The primary use of SFT data is to optimize and improve the performance of pre-trained models on specific tasks. For example, in Natural Language Processing (NLP), SFT data can be a set of dialogues containing questions and answers, used to train a question-answering system. By using SFT data, models can more accurately understand and perform specific tasks, thus performing better in specific domains.

[0022] Fine-tuning is a machine learning technique used to optimize and adjust a pre-trained model to fit a specific task or dataset. In this process, the model continues to be trained using domain-specific data to refine its understanding and responsiveness to that domain. This approach is particularly suitable for models that have already been pre-trained on large, diverse datasets. Through fine-tuning, these models can be better adapted to specific language understanding or generation tasks, such as sentiment analysis, text classification, and question-answering systems. Fine-tuning is a crucial step in bridging extensive pre-training with specific application requirements, helping to improve the model's accuracy and efficiency on specific tasks.

[0023] The challenges of large language models in processing long texts lie in the scarcity of long texts, high computational demands, and significant forgetting of short contextual capabilities. Recent methods attempt to construct long contexts for instruction fine-tuning, but these typically require large language models or human intervention. These approaches are costly and limited in length and diversity. Current approaches to enriching long texts include context-based large language models that modify the model structure and long-context large language models that synthesize data.

[0024] Contextual large language models based on model structure modifications primarily improve performance by adjusting the model structure. Some methods enhance the model's ability to represent positional information by improving positional encoding design. Additionally, there are methods that improve data processing efficiency by optimizing batch processing strategies. Parameter-efficient training focuses on maintaining performance while reducing computational resource consumption. Other methods modify the model architecture of the attention mechanism. However, these methods typically use text of a certain length for training, while long-context training data is relatively scarce. Therefore, constructing high-quality contextual data is particularly important.

[0025] Large-context language models based on data synthesis focus on creating high-quality contextual data for training. However, current data construction methods require the capabilities of large-context language models or human intervention, which leads to high costs. Furthermore, the long text training data generated by large-context language models is limited in terms of data length and diversity.

[0026] This specification provides a method for generating text data, and also relates to a method for training a question-answering model, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail in the following embodiments.

[0027] See Figure 1 , Figure 1 A flowchart of a text data generation method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0028] Step 102: Receive a text data generation instruction for an initial sample text set, wherein the initial sample text set includes multiple initial sample texts.

[0029] The initial sample text set can be a collection of short texts that are currently available, and it includes multiple initial sample texts. Initial sample texts can be understood as texts with a length less than a preset threshold.

[0030] In the methods provided in the embodiments of this specification, multiple long text instruction fine-tuning datasets are constructed. Instruction fine-tuning is relative to model pre-training. During the training process of a question-answering model, pre-training uses a large amount of unlabeled text, allowing the model to predict the next character each time. In one or more embodiments of this specification, the question-answering model can be understood as a large language model. Instruction fine-tuning can also be called supervised fine-tuning, which means that there is a standard answer for the input text, and the model is trained using the standard answer. Typically, instruction fine-tuning is performed after the large language model is pre-trained. After instruction fine-tuning, the large language model can exhibit strong instruction compliance ability and can solve various downstream tasks through zero-shot learning. In the method provided in this specification, it is necessary to train the large language model's ability to process long texts, therefore, multiple long text instruction fine-tuning datasets need to be constructed. Long text instruction fine-tuning datasets are the long text data used for instruction fine-tuning of the large language model.

[0031] Based on this, the text data generation instruction for the initial sample text set specifically refers to the generation instruction for long text instruction fine-tuning data. In this method, an initial sample text set is obtained in advance, and the initial sample text in the initial sample text set is short text instruction fine-tuning data. Using the method provided in the embodiments of this specification, long text instruction fine-tuning data can be generated from the short text instruction fine-tuning data.

[0032] In one specific embodiment provided in this specification, the initial sample text may be: "Instruction: what's the answer of 2+3?; Response: The answer of 2+3 is5".

[0033] "Instruction: Find x for 5x-10=0; Response: The value of x is 2".

[0034] "Instruction: what's the smallest prime number; Response: smallest prime number is 2".

[0035] The above are three examples of initial sample texts. In practical applications, there are many forms of initial sample texts, which are not limited in this specification.

[0036] Step 104: In response to the text data generation instruction, determine at least one text augmentation type and obtain the text augmentation template corresponding to each text augmentation type, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type.

[0037] Technicians have discovered through research that the three most important capabilities of large language models in processing long texts are multi-document retrieval, few-shot learning, and summarization. Other contextual capabilities can be transferred based on these three capabilities. Therefore, the methods provided in the embodiments of this specification require the creation of long text instruction fine-tuning data that can improve these three capabilities of the large language model.

[0038] Based on this, at least one text augmentation type is determined according to the text data generation instruction. The text augmentation types include multi-document retrieval, few-shot learning, and summarization. These correspond to the three capabilities of adjusting a large language model. In the method provided in the embodiments of this specification, in response to the text data generation instruction, at least one text augmentation type is determined. This can be done by randomly selecting at least one of the above three types, or by parsing the text data generation instruction to obtain at least one text augmentation type carried by the text data generation instruction. Furthermore, to generate richer long text instruction fine-tuning data, all three text augmentation types can be determined simultaneously.

[0039] Each text expansion type corresponds to a different text expansion method. In the method provided in the embodiments of this specification, a corresponding text expansion template can be pre-created for each text expansion type. The text expansion template is used to assist in generating fine-tuning data for long text instructions.

[0040] Long text data created based on the multi-document retrieval type can help large language models effectively improve their retrieval capabilities across multiple documents; long text data created based on the few-shot learning type can help large language models effectively improve their ability to learn a specified format from a small number of samples; long text data created based on the summary type can help large language models effectively improve their ability to extract summaries from documents.

[0041] In one specific embodiment provided in this specification, obtaining the text expansion template corresponding to each text expansion type includes: Obtain at least one initial meta-instruction corresponding to the initial text expansion template and the target text expansion type, wherein the target text expansion type is any one of the text expansion types; The target meta-instruction is determined from each initial meta-instruction, and a text expansion template of the target text expansion type is generated based on the initial text expansion template and the target meta-instruction.

[0042] In practical applications, to enhance the diversity of text expansion templates, different text expansion templates can be generated based on pre-set meta-instruction groups.

[0043] In the method provided in the embodiments of this specification, the text augmentation template includes short text instruction fine-tuning data and meta-instructions. The meta-instructions are used to guide the large language model to perform different processing based on multiple short text instruction fine-tuning data and meta-instructions. Format requirements, response order, constraints, etc., for the multiple short text instruction fine-tuning data can be added to the meta-instructions. Different text augmentation types correspond to different text augmentation templates, and the meta-instructions included in different text augmentation templates indicate the capabilities that the text augmentation template can enhance for the large language model.

[0044] For example, for text augmentation templates corresponding to few-shot learning types, the included meta-instructions can specify a fixed format, requiring the large language model to respond according to this fixed format. As another example, for text augmentation modules corresponding to multi-document retrieval types, the included meta-instructions guide the large language model to locate information in multiple documents, helping it find the corresponding document and perform subsequent processing.

[0045] To further enrich the types of text expansion templates, multiple different meta-directions can be pre-set to form an initial meta-direction group for the same text expansion type. The initial meta-direction group includes multiple initial meta-directions.

[0046] In practical applications, there can be multiple ways to express the same text expansion type. If a fixed meta-instruction is used, the generated long text instruction fine-tuning data will be monotonous and inflexible. In the method provided in the embodiments of this specification, multiple initial meta-instructions are pre-configured for the same text expansion type. When generating a text expansion template, a meta-instruction can be randomly selected from the multiple initial meta-instructions, and the selected meta-instruction is the target meta-instruction. The target meta-instruction is combined with the initial text expansion template to obtain the text expansion template.

[0047] The initial text expansion template serves as the foundation for generating templates corresponding to various text expansion types. The target text expansion type can be understood as the text expansion type for which a template needs to be generated. The initial text expansion module stores the fill positions corresponding to meta-instructions. After determining the target meta-instruction corresponding to the target text expansion type, the target meta-instruction is filled into the fill positions in the initial text expansion template, thus generating the text expansion template corresponding to the target text expansion type. This method enhances the diversity of text expansion templates.

[0048] In practical applications, the initial meta-instruction corresponds to each text expansion type, with different text expansion types corresponding to different initial meta-instructions. In another specific embodiment provided in this specification, it also includes: Receive a meta-instruction generation instruction for a target text expansion type, wherein the meta-instruction generation instruction carries expansion semantic information corresponding to the target text expansion type; At least one initial meta-instruction is generated for the target text extension type based on the extended semantic information.

[0049] In this embodiment, the initial meta-instruction is further defined as being created based on the meta-instruction generation instruction for the target text augmentation type. When a technician wishes to generate an initial meta-instruction for a target text augmentation type, they issue a meta-instruction generation instruction for that target text augmentation type. This meta-instruction generation instruction carries augmentation semantic information corresponding to the target text augmentation type. This augmentation semantic information can be understood as the semantic information corresponding to the target text augmentation type. For example, if the target text augmentation type is a multi-document retrieval type, the augmentation semantic information represents enhanced multi-document retrieval capabilities; as another example, if the target text augmentation type is a few-shot learning type, the augmentation semantic information represents enhanced text learning capabilities; and as yet another example, if the target text augmentation type is a summarization type, the augmentation semantic information represents enhanced summarization capabilities. Different text augmentation types correspond to different augmentation semantic information. This augmentation semantic information guides the generation of the initial meta-instruction corresponding to the specific text augmentation type.

[0050] After determining the augmented semantic information, at least one initial meta-instruction can be generated for the target text augmentation type based on this information. Furthermore, the process of generating initial meta-instructions can be implemented using a large language model. The generation of initial meta-instructions for different text augmentation types will be explained below.

[0051] In one specific embodiment provided in this specification, when the target text augmentation type is a multi-document retrieval type, the augmented semantic information is used to represent enhanced multi-document retrieval capabilities; Generate at least one initial meta-instruction for the target text expansion type based on the expanded semantic information, including: Based on the enhanced multi-document retrieval capability, at least one initial first meta-instruction is generated for the multi-document retrieval type, wherein the initial first meta-instruction is used to locate text in at least two initial sample texts.

[0052] When the target text augmentation type is multi-document retrieval, the training focuses on the large language model's document retrieval capabilities across multiple documents. The generated initial meta-instructions are intended to guide the large language model in locating documents within at least two initial sample texts. In this case, the initial first meta-instruction serves as the instruction guiding multi-document retrieval, such as "What is the answer to the second question after question 1?", or "Only questions without answers in the instance need to be answered," and so on. Within the initial first meta-instruction, specific content cannot be directly located; retrieval across multiple documents is required based on the content of the initial first meta-instruction.

[0053] The initial first-order instruction guides the large language model to locate information in the context, learn the positional relationships between different pieces of information, and help the large language model understand the positional relationships of information in a document, identify whether the document contains key information, and so on.

[0054] In another specific embodiment provided in this specification, when the target text augmentation type is a few-shot learning type, the augmented semantic information is used to represent the enhanced text learning ability; Accordingly, at least one initial meta-instruction is generated for the target text expansion type based on the expanded semantic information, including: Based on the enhanced text learning capability, at least one initial second-order instruction is generated for the few-shot learning type, wherein the initial second-order instruction is used to learn examples of the initial sample text.

[0055] In this embodiment, the example of using few-shot learning type for target text augmentation and augmenting semantic information to represent enhanced text learning ability is used for explanation. The initial second-order instruction generated based on the enhanced text learning ability aims to guide the large language model to learn a specified format or example. Therefore, the initial second-order instruction includes examples used by the large language model to learn the initial sample text. For example, the initial second-order instruction could be "Based on the above example, please answer the following question:", or "Please rewrite the following content according to the *** format:", etc. The initial second-order instruction specifies the format of the output content, guiding the large language model to output content according to the specified format.

[0056] In the initial second-order instruction, some examples are selected as minority examples, which enable the large language model to answer new instructions based on these minority examples, thereby improving the understanding and reasoning ability of the large language model.

[0057] In another specific embodiment provided in this specification, when the target text augmentation type is a summary type, the augmented semantic information is used to represent enhanced summarization capabilities; Accordingly, at least one initial meta-instruction is generated for the target text expansion type based on the expanded semantic information, including: Based on the enhanced summarization capability, at least one initial tertiary instruction is generated for the summarization type, wherein the initial tertiary instruction is used to summarize the initial sample text.

[0058] In this embodiment, the explanation is based on the example of using target text augmentation type as the summary type and augmenting semantic information to represent enhanced summarization capabilities. The initial tertiary instruction generated based on the enhanced summarization capabilities aims to improve the large language model's ability to understand documents and extract summaries. The initial tertiary instruction includes a summary of the content of the initial sample text. For example, the initial tertiary instruction could be "***The answer is the answer to question number X above".

[0059] The initial tertiary instruction includes instructions to guide the large language model to summarize the initial sample text, enabling the large language model to understand and summarize the answer while retrieving relevant content from multiple initial sample texts.

[0060] Step 106: Select initial sample text from the initial sample text set and add it to the target text expansion template to generate the target sample text corresponding to the target text expansion template. The target text expansion template is any one of the text expansion templates, and the target sample text is used to train the question answering model.

[0061] After determining the target text expansion template, initial sample text can be selected from the initial sample text set and added to the target text expansion template to generate the target sample text corresponding to the target text expansion template. The target sample text is the long text instruction fine-tuning data, used to train and fine-tune the question-answering model. In the method provided in the embodiments of this specification, the question-answering model is a large language model.

[0062] In the method provided in the embodiments of this specification, an appropriate amount of initial sample text is obtained from an initial sample text set, and then a corresponding target sample text is generated according to the target text expansion template. The number of characters in the target sample text is greater than a preset threshold, that is, the target sample text meets the character count requirement for long texts. Specifically, selecting initial sample text from the initial sample text set and adding it to the target text expansion template includes: Select reference initial sample texts from the initial sample text set and count the number of characters in the sample texts; If the number of characters in the sample text meets the preset character count threshold, the reference initial sample text is added to the target text expansion template so that the number of characters in the target sample text is greater than or equal to the preset character count threshold. If the number of characters in the sample text does not meet the preset character count threshold, the operation of selecting a reference initial sample text from the initial sample text set and counting the number of characters in the sample text continues.

[0063] In this embodiment, a reference initial sample text is randomly selected from the initial sample text set. This reference initial sample text is the initial sample text used to add to the target text expansion template. After determining the reference initial sample text, the number of sample text characters in the selected reference initial sample text is counted. It should be noted that the number of sample text characters here refers to the total number of characters in all selected reference initial sample texts.

[0064] If the number of characters in each selected initial reference text meets the preset character count threshold, then the selected initial reference text is added to the target text expansion template. The resulting target text will then have a character count greater than or equal to the preset character count threshold, satisfying the character requirement for long texts. It's important to note that when adding each initial reference text to the target text expansion template, adjustments are made based on the target meta-instructions within the template to ensure a better fit between the initial reference text and the target meta-instructions.

[0065] If the number of characters in the selected initial reference sample text does not meet the preset character count threshold, then another initial reference sample text will be selected from the initial sample text set, and the number of characters in the sample text will continue to be counted until the preset character count threshold is met.

[0066] In another specific embodiment provided in the embodiments of this specification, adding reference initial sample text to the target text expansion template includes: Obtain the target meta-instructions of the target text augmentation template; According to the target meta-instruction, a reference target sample text and a reference sample answer corresponding to the reference target sample text will be determined from each reference initial sample text; Add the target sample text and the target sample answer to the target text extension template.

[0067] During the process of filling each reference initial sample text into the target text expansion template, the target meta-instructions in the target text expansion template are obtained. Based on the information of the target meta-instructions, the reference target sample text and reference sample answer are determined from each reference initial sample text. The reference target sample text refers to the question information in the target text expansion template, and the reference sample answer refers to the answer information in the target text expansion template.

[0068] Add the reference target sample text and reference sample answer to the target text expansion template to generate the corresponding target sample text.

[0069] Furthermore, the target text expansion template includes a sample text field and a sample answer field; Add the reference target sample text and reference sample answer to the target text expansion template, including: Add the reference target sample text to the sample text field, and add the reference sample answer to the sample answer field.

[0070] In practical applications, the target text expansion template includes a sample text field and a sample answer field. The sample text field is used to add question information, and the sample answer field is used to add answer information. After obtaining the reference target sample text and reference sample answer, the reference target sample text is added to the sample text field, and the reference sample answer is added to the sample answer field.

[0071] For example, the initial sample text might be: "Instruction: What's the answer of 2+3? Response: The answer of 2+3 is 5". "Instruction: Find x for 5x-10=0 Response: The value of x is 2". "Instruction: What's the smallest prime number? Response: The smallest prime number is 2".

[0072] The target text augmentation template is a template designed to enhance multi-document retrieval capabilities. The meta-instruction in the template is "What is the answer to the second question after question 1?". Adding the initial reference sample text to the target text augmentation template yields the target sample text corresponding to the template. Instruction: (1) what's the answer of 2+3? (2) Find x for 5x-10=0? (3) what's the smallest prime number? What is the answer to the second question after question 1? Response: the smallest prime number is 2” The text “Instruction: (1) What's the answer of 2+3? (2) Find x for 5x-10=0? (3) What's the smallest prime number?” is the reference target sample text. “What is the answer to the second question after question 1?” is the meta-instruction. “Response: The smallest prime number is 2” is the reference sample answer.

[0073] For example, using a target text augmentation template to enhance summarization capabilities, where the meta-instruction in the template is "***The answer is the answer to question number X above", adding the initial reference sample text to the target text augmentation template yields the target sample text corresponding to the target text augmentation template. Instruction: (1) what's the answer of 2+3? (2) Find x for 5x-10=0? (3) what's the smallest prime number? What is the question number corresponding to the answer 'the smallest prime number is 2'? Response: 3” The text “Instruction: (1) what's the answer of 2+3? (2) Find x for 5x-10=0? (3) what's the smallest prime number?” is the reference target sample text. “What is the question number corresponding to the answer ‘the smallest prime number is 2’?” is the meta-instruction. “Response: 3” is the reference sample answer.

[0074] For example, using a target text augmentation template to enhance text learning capabilities, the meta-instruction in the template is "Based on the above example, please answer the following question:". Adding the initial reference sample text to the target text augmentation template yields the target sample text corresponding to the target text augmentation template. Instruction: Question(2)Find x for 5x-10=0? Answer(2)The value of x is 2. Question (3) what's the smallest prime number? Answer(3)the smallest prime number is 2. Based on the example above, please answer the following question: What is the smallest prime number? Response: The answer of 2+3 is 5” The text “Instruction: (1) Question (2) Find x for 5x-10=0? Answer (2) The value of x is 2. Question (3) What's the smallest prime number? Answer (3) The smallest prime number is 2.” is the reference target sample text. “Based on the above example, please answer the following question: What's the smallest prime number?” is the meta-instruction. “Response: The answer of 2+3 is 5” is the reference sample answer.

[0075] In another specific embodiment provided in this specification, selecting initial sample text from the initial sample text set and adding it to the target text expansion template includes: Obtain the sample type corresponding to each initial sample text; Cluster the initial sample texts according to the sample type corresponding to each initial sample text to obtain at least one subset of initial sample texts; Select initial sample text from the target initial sample text subset and add the initial sample text to the target text expansion template, where the target initial sample text subset is any one of the initial sample text subsets.

[0076] The method provided in the embodiments of this specification is applicable to various short text instruction fine-tuning datasets. In practical applications, short text instruction fine-tuning datasets may also contain various types of sample types. To better generate long text instruction fine-tuning datasets and unify the data types in long text instruction fine-tuning datasets, the method provided in the embodiments of this specification can also perform clustering based on the sample types of each initial sample text. When adding to the target text expansion template, reference initial sample texts of the same sample type are selected, thereby making the data types in the target sample text more uniform.

[0077] In practical applications, the data in the short text instruction fine-tuning dataset can be classified into mathematical type, code type, and common sense type. Initial sample texts belonging to the mathematical type are grouped together to obtain the initial sample text subset corresponding to the data type. Similarly, initial sample texts belonging to the code type are grouped together to obtain the initial sample text subset corresponding to the code type, and initial sample texts belonging to the common sense type are grouped together to obtain the initial sample text subset corresponding to the common sense type.

[0078] In the process of selecting initial sample text from the initial sample text set and adding it to the target text expansion template, initial sample text can be further selected from the initial sample text subset corresponding to different sample types, thereby improving the text quality of the target sample text.

[0079] In another specific embodiment provided in this specification, the text expansion type also includes formatting type, sorting type, and masking type.

[0080] In addition to the multi-document retrieval type, few-shot learning type, and summary type mentioned above, the methods provided in the embodiments of this specification may also include other existing types, such as the data augmentation method for fine-tuning large models - Mosaic-IT. Mosaic-IT enhances the instruction following ability of a large language model by randomly splicing multiple existing instructions and response data into a whole.

[0081] Formatting type adds format requirements to meta-instructions, forcing the large language model to respond to each instruction in sequence while responding according to the format required in the meta-instructions, thus improving the large language model's format following ability to a certain extent.

[0082] The sorting type adds a requirement for the order of responses to the meta-instructions. Under this strategy, the large language model no longer simply responds according to the given order of instructions, but needs to respond according to the order defined in the meta-instructions. This process forces the large model to first determine the correct response order and then respond step by step.

[0083] Masking type adds a masking requirement for the response to the meta-instruction. Under this strategy, the large semantic model needs to ignore some instructions in the meta-instruction, which further improves the large language model's understanding of the complex dependencies between instructions.

[0084] The method provided in this specification, based on an initial sample text set, aims to generate target sample text by producing long text instruction fine-tuning data for training a large language model, enhancing its multi-document retrieval capabilities, few-shot learning capabilities, and summarization capabilities. During the generation process, different text augmentation types are determined based on different capabilities, and text augmentation templates corresponding to different text augmentation types are obtained. Initial sample texts from the initial sample text set are randomly added to the text augmentation templates to generate the corresponding target sample texts. Utilizing the initial sample text set to generate different types of long text instruction fine-tuning data enriches the diversity of long text instruction fine-tuning data and eliminates the need for large language models or manual intervention, saving data acquisition costs. During the selection of reference initial sample texts from the initial sample text set, the number of characters in the sample texts is also counted. When the number of characters in a sample text meets a preset character count threshold, it is added to the target text augmentation template, ensuring that the length of the final generated target sample text is controllable.

[0085] The following is in conjunction with the appendix Figure 2 Taking the application of the text data generation method provided in this specification in generating fine-tuning data for long text instructions as an example, the text data generation method will be further explained. Among them, Figure 2 The flowchart of a method for generating fine-tuning data for long text instructions provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0086] Step 202: Obtain the initial sample text set, which includes multiple initial sample texts.

[0087] Step 204: Receive a text data generation instruction for the initial sample text set, and the text data generation instruction determines at least one text augmentation type.

[0088] Step 206: Determine multiple data augmentation strategies based on each text augmentation type. These strategies include few-sample question-and-answer strategies, before-and-after-answer strategies, answer-without-answer strategies, answer-to-ID strategies, and formatting, sorting, and masking strategies in mosaic instruction tuning.

[0089] Among them, the few-shot question answering strategy is used to enhance the few-shot learning ability of large language models. This strategy selects some instructions as few-shot examples in the meta-instructions and requires the large language model to answer new instructions based on these examples. The long text instruction fine-tuning data generated by this strategy can improve the understanding and reasoning ability of large language models.

[0090] The pre- and post-answer strategy is used to improve the multi-document retrieval capabilities of large language models. This strategy guides large language models to determine the relevance of information based on its position in the context, such as answering questions before or after a certain question, thereby helping large language models understand the positional relationships of information in documents.

[0091] The "without answer" strategy is also used to improve the multi-document retrieval capabilities of large language models. This strategy concatenates multiple questions, i.e., a preset number of answers, into a single instruction, guiding the large language model to only answer the remaining questions that have no answers. In this way, the large language model can learn to identify whether a document contains key information.

[0092] The answer-to-ID strategy is used to improve the multi-document retrieval and summarization capabilities of large language models. This strategy randomly connects multiple questions without attaching answers, allowing the large language model to identify the ID of the corresponding question based on the given answer. This enables the large language model to retrieve related questions from multiple questions while understanding and summarizing the answers.

[0093] The formatting strategy, sorting strategy, and masking strategy are all strategies mentioned in the mosaic instruction tuning, and are not limited in the embodiments of this specification.

[0094] It should be noted that the data augmentation strategies mentioned in this embodiment are all aimed at improving the multi-document retrieval capability, few-shot learning capability, and summarization capability of the large language model. The embodiments in this specification are merely illustrative and do not limit the number or form of data augmentation strategies. Data augmentation strategies only need to satisfy the requirement of improving the relevant capabilities of the large language model.

[0095] Step 208: Obtain the text augmentation templates corresponding to each data augmentation strategy.

[0096] Step 210: Divide each initial sample text in the initial sample text set into multiple initial sample text subsets according to the sample type.

[0097] Step 212: Select at least one reference initial sample text from the target initial sample text subset, and generate long text instruction fine-tuning data based on each reference initial sample text and each text expansion template.

[0098] Corresponding to the above method embodiments, this specification also provides embodiments of a text data generation apparatus. Figure 3 A schematic diagram of a text data generation apparatus according to one embodiment of this specification is shown. Figure 3 As shown, the device includes: The receiving module 302 is configured to receive a text data generation instruction for an initial sample text set, wherein the initial sample text set includes multiple initial sample texts; The acquisition module 304 is configured to determine at least one text augmentation type in response to the text data generation instruction, and acquire the text augmentation template corresponding to each text augmentation type, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type; The generation module 306 is configured to select initial sample text from the initial sample text set, add it to the target text expansion template, and generate target sample text corresponding to the target text expansion template. The target text expansion template is any one of the text expansion templates, and the target sample text is used to train the question answering model.

[0099] Optionally, the acquisition module 304 is further configured to: Obtain at least one initial meta-instruction corresponding to the initial text expansion template and the target text expansion type, wherein the target text expansion type is any one of the text expansion types; The target meta-instruction is determined from each initial meta-instruction, and a text expansion template of the target text expansion type is generated based on the initial text expansion template and the target meta-instruction.

[0100] Optionally, the device further includes a meta-instruction generation module, configured to: Receive a meta-instruction generation instruction for a target text expansion type, wherein the meta-instruction generation instruction carries expansion semantic information corresponding to the target text expansion type; At least one initial meta-instruction is generated for the target text extension type based on the extended semantic information.

[0101] Optionally, when the target text augmentation type is a multi-document retrieval type, the augmented semantic information is used to represent enhanced multi-document retrieval capabilities; The meta-instruction generation module is further configured as follows: Based on the enhanced multi-document retrieval capability, at least one initial first meta-instruction is generated for the multi-document retrieval type, wherein the initial first meta-instruction is used to locate text in at least two initial sample texts.

[0102] Optionally, when the target text augmentation type is a few-shot learning type, the augmented semantic information is used to represent the enhanced text learning ability; The meta-instruction generation module is further configured as follows: Based on the enhanced text learning capability, at least one initial second-order instruction is generated for the few-shot learning type, wherein the initial second-order instruction is used to learn examples of the initial sample text.

[0103] Optionally, when the target text augmentation type is a summary type, the augmented semantic information is used to represent enhanced summarization capabilities; The meta-instruction generation module is further configured as follows: Based on the enhanced summarization capability, at least one initial tertiary instruction is generated for the summarization type, wherein the initial tertiary instruction is used to summarize the initial sample text.

[0104] Optionally, the generation module 306 is further configured to: Select reference initial sample texts from the initial sample text set and count the number of characters in the sample texts; If the number of characters in the sample text meets the preset character count threshold, the reference initial sample text is added to the target text expansion template so that the number of characters in the target sample text is greater than or equal to the preset character count threshold. If the number of characters in the sample text does not meet the preset character count threshold, the operation of selecting a reference initial sample text from the initial sample text set and counting the number of characters in the sample text continues.

[0105] Optionally, the generation module 306 is further configured to: Obtain the target meta-instructions of the target text augmentation template; According to the target meta-instruction, a reference target sample text and a reference sample answer corresponding to the reference target sample text will be determined from each reference initial sample text; Add the target sample text and the target sample answer to the target text extension template.

[0106] Optionally, the target text expansion template includes a sample text field and a sample answer field; The generation module 306 is further configured to: Add the reference target sample text to the sample text field, and add the reference sample answer to the sample answer field.

[0107] Optionally, the generation module 306 is further configured to: Obtain the sample type corresponding to each initial sample text; Cluster the initial sample texts according to the sample type corresponding to each initial sample text to obtain at least one subset of initial sample texts; Select initial sample text from the target initial sample text subset and add the initial sample text to the target text expansion template, where the target initial sample text subset is any one of the initial sample text subsets.

[0108] Optionally, the text expansion types also include formatting types, sorting types, and masking types.

[0109] The apparatus provided in this specification, based on an initial sample text set, aims to generate target sample text by producing long text instruction fine-tuning data for training a large language model, enhancing its multi-document retrieval capabilities, few-shot learning capabilities, and summarization capabilities. During the generation process, different text expansion types are determined based on different capabilities, and text expansion templates corresponding to different text expansion types are obtained. Initial sample texts from the initial sample text set are randomly added to the text expansion templates to generate the corresponding target sample texts. Utilizing the initial sample text set to generate different types of long text instruction fine-tuning data enriches the diversity of long text instruction fine-tuning data and eliminates the need for large language models or manual intervention, saving data acquisition costs. During the selection of reference initial sample texts from the initial sample text set, the number of characters in the sample text is also counted. When the number of characters in the sample text meets a preset character count threshold, it is added to the target text expansion template, ensuring that the length of the final generated target sample text is controllable.

[0110] The above is an illustrative scheme of a text data generation device according to this embodiment. It should be noted that the technical solution of this text data generation device and the technical solution of the text data generation method described above belong to the same concept. For details not described in detail in the technical solution of the text data generation device, please refer to the description of the technical solution of the text data generation method described above.

[0111] Figure 4 A flowchart of a text data generation method for cloud-side devices provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0112] Step 402: Receive the text data generation instruction sent by the receiving end device.

[0113] Step 404: Obtain the initial sample text set corresponding to the text data generation instruction, wherein the initial sample text set includes multiple initial sample texts.

[0114] Step 406: In response to the text data generation instruction, determine at least one text augmentation type and obtain the text augmentation template corresponding to each text augmentation type, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type.

[0115] Step 408: Select initial sample text from the initial sample text set and add it to the target text expansion template to generate the target sample text corresponding to the target text expansion template. The target text expansion template is any one of the text expansion templates, and the target sample text is used to train the question answering model.

[0116] Step 410: Send the text of each target sample to the end device.

[0117] In this embodiment, the text data generation method can also be deployed on cloud-side devices. If the configuration of the terminal device is low, there may be issues such as slow processing speed and lag when running the text data generation method. In this embodiment, deploying the method on a cloud-side device can make full use of the ample computing resources on the cloud-side device, expand the application scenarios of the method, and enable terminals with lower configurations to use the method to obtain diverse target sample texts, thereby improving the user experience.

[0118] Figure 5 A flowchart of a training method for a question-answering model applied to a cloud-side device, according to an embodiment of this specification, is shown, specifically including the following steps.

[0119] Step 502: Receive the text data generation instruction sent by the receiving end device.

[0120] Step 504: Obtain the initial sample text set corresponding to the text data generation instruction, wherein the initial sample text set includes multiple initial sample texts.

[0121] Step 506: In response to the text data generation instruction, determine at least one text augmentation type and obtain the text augmentation template corresponding to each text augmentation type, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type.

[0122] Step 508: Select initial sample text from the initial sample text set, add it to the target text expansion template, and generate the target sample text corresponding to the target text expansion template, wherein the target text expansion template is any one of the text expansion templates.

[0123] Step 510: Train a question-answering model based on the target sample text, obtain the model parameters of the question-answering model, and send the model parameters to the end device.

[0124] In this embodiment, if the question-answering model is a large language model, due to the large number of model parameters and the large model size, there are certain requirements for the terminal deploying the large language model. If the terminal configuration is low, it may not be able to deploy and run the large language model and perform the corresponding processing. Therefore, in this embodiment, the question-answering model can be deployed on a cloud-side device. In actual application, as long as the cloud-side device receives the text data generation instruction sent by the terminal device, it can execute the above-mentioned text data generation method according to the text data generation instruction provided in the embodiments of this specification to obtain the corresponding target sample text, then train the question-answering model using the target sample text, and feed back the model parameters of the question-answering model to the terminal device.

[0125] The training method for the question-answering model running on cloud-side devices expands the application scenarios of this method, enabling even low-configuration terminals to use this method to obtain diverse target sample texts, train the question-answering model based on the target sample texts, and send the model parameters of the trained question-answering model to the edge device so that the edge device can reproduce the question-answering model according to the model parameters, thereby improving the user experience.

[0126] See Figure 6 , Figure 6 This specification illustrates an architecture diagram of a text data generation system according to an embodiment of the present specification. The text data generation system may include a client 100 and a server 200. Client 100 is used to send a text data generation instruction for an initial sample text set to server 200, wherein the initial sample text set includes multiple initial sample texts; Server 200 is configured to determine at least one text augmentation type in response to the text data generation instruction, and obtain text augmentation templates corresponding to each text augmentation type, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type; select initial sample text from the initial sample text set, add it to the target text augmentation template, and generate target sample text corresponding to the target text augmentation template, wherein the target text augmentation template is any one of the text augmentation templates, and the target sample text is used to train the question answering model; and send the target sample text to client 100. Client 100 is also used to receive target sample text sent by server 200.

[0127] The text data generation system may include multiple clients 100 and a server 200. Clients 100 can be referred to as edge devices, and server 200 can be referred to as cloud devices. Multiple clients 100 can establish communication connections through server 200. In the text data generation scenario, server 200 is used to provide text data generation services between multiple clients 100. Each client 100 can act as a sender or receiver, communicating through server 200.

[0128] Users can interact with server 200 through client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In the text data generation scenario, users can publish data streams to server 200 through client 100, server 200 can generate target sample text based on the data stream, and push the target sample text to other clients that have established communication.

[0129] In this system, client 100 and server 200 establish a connection via a network. The network provides the medium for communication between client 100 and server 200. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. Data transmitted by client 100 may need to undergo encoding, transcoding, compression, or other processing before being published to server 200.

[0130] Client 100 can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. Client 100 can be developed based on the software development kit (SDK) of the corresponding service provided by server 200, such as a real-time communication (RTC) SDK. Client 100 can be deployed on a computing device and depends on the device or certain apps on the device to run. The computing device may have a display screen and support information browsing, such as a personal mobile terminal like a mobile phone, tablet, or personal computer. Various other types of applications can also be configured on the computing device, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0131] Server 200 may include servers providing various services, such as servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It should be noted that server 200 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0132] It is worth noting that the text data generation method provided in the embodiments of this specification is generally executed by the server. However, in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the text data generation method provided in the embodiments of this specification. In other embodiments, the text data generation method provided in the embodiments of this specification may also be executed jointly by the client and the server.

[0133] Figure 7 A structural block diagram of a computing device 700 according to an embodiment of this application is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0134] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0135] In one embodiment of this application, the aforementioned components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0136] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 700 can also be a mobile or stationary server.

[0137] The processor 720 is used to execute the following computer program / instruction, which, when executed by the processor, implements the steps of the above-mentioned text data generation method and question-answering model training method.

[0138] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solutions of the above-mentioned text data generation method and question answering model training method. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solutions of the above-mentioned text data generation method and question answering model training method.

[0139] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described text data generation method and question-answering model training method.

[0140] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computer-readable storage medium embodiment is described simply because it is substantially similar to the text data generation method and the question-answering model training method embodiment. Relevant details can be found in the descriptions of the text data generation method and the question-answering model training method embodiments.

[0141] An embodiment of this specification also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described text data generation method and question-answering model training method.

[0142] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solutions of the above-mentioned text data generation method and question-answering model training method. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solutions of the above-mentioned text data generation method and question-answering model training method.

[0143] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0144] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0145] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0146] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0147] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for generating text data, comprising: Receive a text data generation instruction for an initial sample text set, wherein the initial sample text set includes multiple initial sample texts; In response to the text data generation instruction, at least one text augmentation type is determined, and a text augmentation template corresponding to each text augmentation type is obtained, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type; Initial sample text is selected from the initial sample text set and added to the target text expansion template to generate the target sample text corresponding to the target text expansion template. The target text expansion template is any one of the text expansion templates, and the target sample text is used to train the question answering model.

2. The method as described in claim 1, wherein obtaining the text expansion template corresponding to each text expansion type includes: Obtain at least one initial meta-instruction corresponding to the initial text expansion template and the target text expansion type, wherein the target text expansion type is any one of the text expansion types; The target meta-instruction is determined from each initial meta-instruction, and a text expansion template of the target text expansion type is generated based on the initial text expansion template and the target meta-instruction.

3. The method of claim 2, further comprising: Receive a meta-instruction generation instruction for a target text expansion type, wherein the meta-instruction generation instruction carries expansion semantic information corresponding to the target text expansion type; At least one initial meta-instruction is generated for the target text extension type based on the extended semantic information.

4. The method as described in claim 3, wherein when the target text augmentation type is a multi-document retrieval type, the augmented semantic information is used to represent enhanced multi-document retrieval capabilities; Generate at least one initial meta-instruction for the target text expansion type based on the expanded semantic information, including: Based on the enhanced multi-document retrieval capability, at least one initial first meta-instruction is generated for the multi-document retrieval type, wherein the initial first meta-instruction is used to locate text in at least two initial sample texts.

5. The method as described in claim 3, wherein when the target text augmentation type is a few-shot learning type, the augmented semantic information is used to represent the enhanced text learning ability; Generate at least one initial meta-instruction for the target text expansion type based on the expanded semantic information, including: Based on the enhanced text learning capability, at least one initial second-order instruction is generated for the few-shot learning type, wherein the initial second-order instruction is used to learn examples of the initial sample text.

6. The method of claim 3, wherein when the target text augmentation type is a summary type, the augmented semantic information is used to represent enhanced summarization capabilities; Generate at least one initial meta-instruction for the target text expansion type based on the expanded semantic information, including: Based on the enhanced summarization capability, at least one initial tertiary instruction is generated for the summarization type, wherein the initial tertiary instruction is used to summarize the initial sample text.

7. The method of claim 1, wherein selecting initial sample text from the initial sample text set and adding it to the target text expansion template comprises: Select reference initial sample texts from the initial sample text set and count the number of characters in the sample texts; If the number of characters in the sample text meets the preset character count threshold, the reference initial sample text is added to the target text expansion template so that the number of characters in the target sample text is greater than or equal to the preset character count threshold. If the number of characters in the sample text does not meet the preset character count threshold, the operation of selecting a reference initial sample text from the initial sample text set and counting the number of characters in the sample text continues.

8. The method of claim 7, wherein adding the reference initial sample text to the target text expansion template includes: Obtain the target meta-instructions of the target text augmentation template; According to the target meta-instruction, a reference target sample text and a reference sample answer corresponding to the reference target sample text will be determined from each reference initial sample text; Add the target sample text and the target sample answer to the target text extension template.

9. The method of claim 8, wherein the target text expansion template includes a sample text field and a sample answer field; Add the reference target sample text and reference sample answer to the target text expansion template, including: Add the reference target sample text to the sample text field, and add the reference sample answer to the sample answer field.

10. The method of claim 1, wherein selecting initial sample text from the initial sample text set and adding it to the target text expansion template comprises: Obtain the sample type corresponding to each initial sample text; Cluster the initial sample texts according to the sample type corresponding to each initial sample text to obtain at least one subset of initial sample texts; Select initial sample text from the target initial sample text subset and add the initial sample text to the target text expansion template, where the target initial sample text subset is any one of the initial sample text subsets.

11. The method according to any one of claims 1-10, wherein the text expansion type further includes a formatting type, a sorting type, and a masking type.

12. A text data generation method, applied to cloud-side devices, comprising: Text data generation instructions sent by the receiving end device; Obtain the initial sample text set corresponding to the text data generation instruction, wherein the initial sample text set includes multiple initial sample texts; In response to the text data generation instruction, at least one text augmentation type is determined, and a text augmentation template corresponding to each text augmentation type is obtained, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type; Initial sample text is selected from the initial sample text set and added to the target text expansion template to generate the target sample text corresponding to the target text expansion template. The target text expansion template is any one of the text expansion templates, and the target sample text is used to train the question answering model. Each target sample text is sent to the terminal device.

13. A training method for a question-answering model, applied to cloud-based devices, comprising: Text data generation instructions sent by the receiving end device; Obtain the initial sample text set corresponding to the text data generation instruction, wherein the initial sample text set includes multiple initial sample texts; In response to the text data generation instruction, at least one text augmentation type is determined, and a text augmentation template corresponding to each text augmentation type is obtained, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type; Initial sample text is selected from the initial sample text set and added to the target text expansion template to generate the target sample text corresponding to the target text expansion template, wherein the target text expansion template is any one of the text expansion templates; A question-answering model is trained based on the target sample text, the model parameters of the question-answering model are obtained, and the model parameters are sent to the edge device.

14. A text data generation system, comprising a client and a server; wherein, The client is configured to send a text data generation instruction to the server for an initial sample text set, wherein the initial sample text set includes multiple initial sample texts; The server is configured to respond to the text data generation instruction to determine at least one text augmentation type and obtain text augmentation templates corresponding to each text augmentation type, wherein the text augmentation types include multi-document retrieval type, few-shot learning type, and summary type; select initial sample text from the initial sample text set and add it to the target text augmentation template to generate target sample text corresponding to the target text augmentation template, wherein the target text augmentation template is any one of the text augmentation templates, and the target sample text is used to train the question answering model; and send the target sample text to the client.

15. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 13.

16. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

17. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

Images