Text generation method and device, model training method and device and electronic equipment

Abstract

The embodiment of the invention discloses a text generation method and device, a model training method and device and electronic equipment, and the method comprises the steps: obtaining an initial instruction and a constraint dimension text, calling a first large language model, adding various constraint condition texts into the initial instruction based on the constraint dimension text, and outputting a first instruction; calling the first large language model to delete at least one constraint condition text from the first instruction, and outputting a second instruction; calling the first large language model to perform text generation based on the first instruction and the second instruction, and outputting sample texts corresponding to the first instruction and the second instruction; and combining the sample texts to obtain a comparison text set, and training the second large language model based on the comparison text set, so that the instruction following capability of the large language model can be improved, and when the target instruction is input into the trained second large language model, the quality of the generated target text can be effectively improved. The method can be widely applied to scenes such as cloud technology, artificial intelligence and intelligent traffic.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a text generation method, a model training method, a device, and an electronic device. Background Art

[0002] With the development of artificial intelligence technology, large language models (LLMs) can now be used for text generation. Typically, text generation instructions are input into a trained LLM, which then generates the corresponding target text based on the text generation instructions. However, in related technologies, large language models have poor ability to follow text generation instructions, resulting in low-quality target text. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail in this application. This overview is not intended to limit the scope of protection of the claims.

[0004] The embodiments of the present application provide a text generation method, a model training method, an apparatus, and an electronic device, which can improve the instruction-following capability of a large language model and improve the quality of the generated target text.

[0005] In one aspect, an embodiment of the present application provides a text generation method, comprising:

[0006] Obtaining an initial instruction and a constraint dimension text, calling a first language model to add a plurality of constraint condition texts to the initial instruction based on the constraint dimension text, and outputting a first instruction;

[0007] calling the first language model to delete at least one constraint text from the first instruction, and outputting a second instruction;

[0008] Calling the first language model to generate text based on the first instruction and the second instruction, and outputting sample texts corresponding to the first instruction and the second instruction respectively;

[0009] Combining the sample texts to obtain a comparison text set, and training a second language model based on the comparison text set;

[0010] Obtain a target instruction, call the trained second language model to generate text based on the target instruction, and output the target text.

[0011] On the other hand, an embodiment of the present application further provides a text generation device, comprising:

[0012] A first instruction generation module is configured to obtain an initial instruction and a constraint dimension text, invoke a first language model to add a plurality of constraint condition texts to the initial instruction based on the constraint dimension text, and output a first instruction;

[0013] a first instruction rewriting module, configured to call the first large language model to delete at least one constraint text from the first instruction and output a second instruction;

[0014] A first sample generation module is configured to call the first language model to generate text based on the first instruction and the second instruction, and output sample texts corresponding to the first instruction and the second instruction respectively;

[0015] A first training module is configured to combine the sample texts to obtain a comparison text set, and train a second language model based on the comparison text set;

[0016] The text generation module is used to obtain the target instruction, call the trained second language model to generate text based on the target instruction, and output the target text.

[0017] Furthermore, the first language model outputs multiple types of the second instructions, where different types of the second instructions are obtained by deleting different amounts of the constraint text. The first training module is specifically configured to:

[0018] Taking each of the sample texts as a reference text, combining the reference text with at least one remaining sample text to obtain a comparison document set;

[0019] The number of the constraint condition texts satisfied by the reference text is greater than the number of the constraint condition texts satisfied by the remaining sample texts.

[0020] Furthermore, the comparison document set includes the benchmark text and one of the remaining sample texts, and the first training module is further configured to:

[0021] Determining a first reward score for a second language model based on the benchmark text, determining a second reward score for the second language model based on the remaining sample text, and determining a first ratio between the first reward score and the second reward score;

[0022] Determining a third reward score of a third language model based on the benchmark text, determining a fourth reward score of the third language model based on the remaining sample text, and determining a second ratio between the third reward score and the fourth reward score;

[0023] A first loss is determined according to the first ratio and the second ratio, and the second large language model is trained based on the first loss.

[0024] Furthermore, the comparison document set includes the reference text and the remaining plurality of sample texts, and the first training module is further configured to:

[0025] Determining a first reward score for a second language model based on the benchmark text, determining a second reward score for the second language model based on each of the remaining sample texts, and determining a third ratio between the first reward score and a sum of the plurality of second reward scores;

[0026] Determining a third reward score of a third language model based on the benchmark text, determining a fourth reward score of the third language model based on each of the remaining sample texts, and determining a fourth ratio between the third reward score and a sum of the plurality of fourth reward scores;

[0027] A second loss is determined according to the third ratio and the fourth ratio, and the second large language model is trained based on the second loss.

[0028] Furthermore, the first instruction generation module is specifically configured to:

[0029] Determining the constraint weights corresponding to the constraint dimension texts, and constructing a third instruction based on the initial instruction, the constraint dimension texts, and the constraint weights, wherein the third instruction is used to prompt the user to add multiple constraint condition texts to the initial instruction;

[0030] The third instruction is input into the first large language model for text generation, and the first instruction is output.

[0031] Furthermore, the first instruction generation module is further configured to:

[0032] Encoding the initial instruction and the constraint dimension text respectively to obtain a first embedding vector of the initial instruction and a second embedding vector of each constraint dimension text;

[0033] Concatenate the first embedding vector with each of the second embedding vectors to obtain multiple target embedding vectors;

[0034] Calling the weight determination model to perform regression processing on each of the target embedding vectors to obtain the constraint weight corresponding to each of the constraint dimension texts;

[0035] The first training module is further used to:

[0036] The second largest language model and the weight determination model are jointly trained based on the comparison text set.

[0037] Furthermore, the constraint dimension text has multiple levels, and the first instruction generation module is further configured to:

[0038] Determine a first dimensional text from each of the constraint dimensional texts at the first level, and determine a second dimensional text from the constraint dimensional text at the Nth level associated with the first dimensional text, where N is an integer, N≥2;

[0039] The first language model is called to add a plurality of constraint condition texts to the initial instruction based on the second dimensional text, and a first instruction is output.

[0040] Furthermore, the first instruction generation module is further configured to:

[0041] Performing semantic recognition on the initial instruction to obtain a first semantic label of the initial instruction;

[0042] Obtaining a second semantic tag preset for each of the constraint dimension texts at the first level, and matching the second semantic tag with the first semantic tag;

[0043] When the second semantic tag is consistent with the first semantic tag, the constraint dimensional text is determined to be the first dimensional text.

[0044] Furthermore, the text generation device further includes a verification module. After calling the first language model to generate text based on the first instruction and the second instruction respectively and outputting sample texts corresponding to the first instruction and the second instruction respectively, the verification module is specifically used to:

[0045] Calling the first language model to verify the sample text, and outputting a verification result and an explanation text;

[0046] The verification result is used to indicate the integrity of the sample text, and the explanation text is used to explain the verification result.

[0047] Furthermore, the text generation device further includes a text refinement module. After calling the first language model to verify the first instruction and outputting the verification result and the interpretation text, the text refinement module is specifically configured to:

[0048] When the verification result indicates that the sample text is incomplete, the first language model is called to regenerate the text based on the first instruction and the second instruction respectively until the complete sample text is output.

[0049] On the other hand, the embodiment of the present application also provides a model training method, including:

[0050] Obtaining an initial instruction and a constraint dimension text, calling a first language model to add a plurality of constraint condition texts to the initial instruction based on the constraint dimension text, and outputting a first instruction;

[0051] calling the first language model to delete at least one constraint text from the first instruction, and outputting a second instruction;

[0052] Calling the first language model to generate text based on the first instruction and the second instruction, and outputting sample texts corresponding to the first instruction and the second instruction respectively;

[0053] The sample texts are combined to obtain a comparison text set, and a second language model is trained based on the comparison text set.

[0054] On the other hand, an embodiment of the present application further provides a model training device, comprising:

[0055] A second instruction generation module is configured to obtain an initial instruction and a constraint dimension text, invoke the first language model to add a plurality of constraint condition texts to the initial instruction based on the constraint dimension text, and output a first instruction;

[0056] a second instruction rewriting module, configured to call the first large language model to delete at least one constraint text from the first instruction and output a second instruction;

[0057] A second sample generation module is configured to call the first language model to generate text based on the first instruction and the second instruction, and output sample texts corresponding to the first instruction and the second instruction respectively;

[0058] The second training module is used to combine the sample texts to obtain a comparison text set, and train the second largest language model based on the comparison text set.

[0059] On the other hand, an embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned text generation method or model training method when executing the computer program.

[0060] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above-mentioned text generation method or model training method.

[0061] In another aspect, embodiments of the present application further provide a computer program product, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to implement the above-described text generation method or model training method.

[0062] The embodiments of the present application include at least the following beneficial effects: by obtaining an initial instruction and a constraint dimension text, calling the first language model to add multiple constraint texts to the initial instruction based on the constraint dimension text, thereby outputting a more complex first instruction, then calling the first language model to delete at least one constraint text from the first instruction, outputting a second instruction, calling the first language model to generate text based on the first instruction and the second instruction respectively, and outputting sample texts corresponding to the first instruction and the second instruction respectively. Since the complicated first instruction is more accurate and specific, the quality of the generated sample text is better, and since the second instruction contains fewer types of constraint texts than the first instruction, the quality of the generated sample text is slightly worse. Therefore, by combining the sample texts, a comparative text set consisting of sample texts with different qualities can be obtained. Subsequently, the second language model can be trained based on the comparative text set, so that the second language model learns a better quality text generation method, thereby improving the second language model's ability to follow instructions. Subsequently, when the target instruction is obtained and the trained second language model is called to generate text based on the target instruction, the quality of the generated target text can be effectively improved.

[0063] Other features and advantages of the present application will be set forth in the following description, and in part will be apparent from the description, or may be understood by practicing the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0065] Figure 1 An optional schematic diagram of an implementation environment provided for an embodiment of the present application;

[0066] Figure 2a An optional schematic diagram of the text generation method provided in the embodiment of the present application applied in a content creation and editing scenario;

[0067] Figure 2b An optional schematic diagram of the text generation method provided in the embodiment of the present application applied in the scenario of an intelligent question-answering system;

[0068] Figure 3 An optional flowchart of the text generation method provided in the embodiment of the present application;

[0069] Figure 4 An optional schematic diagram of complicating instructions through prompt input provided in an embodiment of the present application;

[0070] Figure 5 An optional schematic diagram of rewriting instructions through prompt input provided in an embodiment of the present application;

[0071] Figure 6 An optional schematic diagram of a restraint system provided in an embodiment of the present application;

[0072] Figure 7 An optional schematic diagram of an embodiment of the present application for complicating instructions by prompting constraint weights in input;

[0073] Figure 8 An optional schematic diagram of the process of outputting a first instruction by assigning constraint weights to texts of different constraint dimensions provided in an embodiment of the present application;

[0074] Figure 9 An optional schematic diagram of refining a sample text through prompt input provided in an embodiment of the present application;

[0075] Figure 10 An optional schematic diagram of performing comparative learning on the second language model provided in an embodiment of the present application;

[0076] Figure 11 An optional schematic diagram of constructing a ranking list to train the second largest language model provided in an embodiment of the present application;

[0077] Figure 12 Another optional schematic diagram of performing comparative learning on the second language model provided in an embodiment of the present application;

[0078] Figure 13 An optional schematic diagram of the complete training process of the text generation method provided in an embodiment of the present application;

[0079] Figure 14 An optional flowchart of the model training method provided in the embodiment of the present application;

[0080] Figure 15 A schematic diagram of an optional structure of the text generation device provided in an embodiment of the present application;

[0081] Figure 16 A schematic diagram of an optional structure of the model training device provided in an embodiment of the present application;

[0082] Figure 17 A partial structural block diagram of a terminal provided in an embodiment of the present application;

[0083] Figure 18 A partial structural block diagram of the server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0085] It should be noted that, in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the characteristics of the target object such as target object attribute information or attribute information set, the permission or consent of the target object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. Among them, the target object can be a user. In addition, when the embodiment of the present application needs to obtain target object attribute information, the target object's separate permission or separate consent will be obtained by means of a pop-up window or jumping to a confirmation page. After clearly obtaining the target object's separate permission or separate consent, the necessary target object-related data for enabling the normal operation of the embodiment of the present application will be obtained.

[0086] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0087] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0088] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive field of computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI domains. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0089] Large language models (LLMs) are deep learning models trained using large amounts of text data. They can generate natural language text or understand the meaning of text. Large language models can handle a variety of natural language tasks, such as text classification, question-answering, and conversation, and are a key path to artificial intelligence.

[0090] Prompts are used to instruct the large language model to perform actions or generate outputs, that is, to instruct the model what actions it should take or what output it should generate when performing a specific task.

[0091] Supervised Fine-Tuning (SFT) is a deep learning method that is usually used on pre-trained large language models. It uses labeled data to fine-tune the model to adapt it to a specific task or domain.

[0092] Contrastive learning is a self-supervised deep learning method that learns useful feature representations by comparing different input samples. It does not require labels, reducing the need for manual annotation. By minimizing a contrastive loss function (such as cosine similarity or Euclidean distance), the network learns to distinguish between different samples. In recent years, contrastive learning has achieved significant progress in fields such as image recognition, speech recognition, and natural language processing.

[0093] With the popularity of large language models, they can be used for text generation. Typically, text generation instructions are input into a trained large language model, which then generates the corresponding target text based on the text generation instructions. However, in related technologies, large language models have poor ability to follow text generation instructions, resulting in low-quality generated target text.

[0094] Based on this, the embodiments of the present application provide a text generation method, a model training method, an apparatus, and an electronic device, which can improve the instruction-following capability of a large language model and improve the quality of the generated target text.

[0095] It should be noted that in the embodiment of the present application, SFT will not only learn one correct answer to an instruction each time, but will introduce negative examples for comparison, so that the model can easily learn a fixed context and increase the local understanding of each constraint.

[0096] Reference Figure 1 , Figure 1 An optional schematic diagram of an implementation environment provided for an embodiment of the present application, the implementation environment includes a terminal 101 and a server 102, wherein the terminal 101 and the server 102 are connected via a communication network.

[0097] Exemplarily, during the model training phase, the server 102 can obtain the initial instruction and constraint dimension text sent by the terminal 101, call the first language model to add multiple constraint condition texts to the initial instruction based on the constraint dimension text, and output the first instruction; call the first language model to delete at least one constraint condition text from the first instruction, and output the second instruction; call the first language model to generate text based on the first instruction and the second instruction respectively, and output sample texts corresponding to the first instruction and the second instruction respectively; combine the sample texts to obtain a comparison text set, and train the second language model based on the comparison text set.

[0098] For example, during the text generation phase, the server 102 may obtain the target instruction sent by the terminal 101, call the trained second language model to generate text based on the target instruction, and output the target text.

[0099] The terminal 101 may be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, etc., but is not limited thereto. The terminal 101 and the server 102 may be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment of the present application.

[0100] Server 102 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Furthermore, server 102 can be a node server in a blockchain network.

[0101] For example, the text generation method in the embodiment of the present application is applicable to a variety of specific application scenarios, such as content creation and editing, intelligent question-answering system, etc. Among them:

[0102] (1) Content creation and editing scenarios.

[0103] Reference Figure 2a , Figure 2a An optional schematic diagram of the text generation method provided in the embodiment of the present application applied in the content creation and editing scenario. In the fields of content creation such as news, advertising, and literary works, the embodiment of the present application can help creators generate texts that meet specific requirements, such as specifying themes, styles, emotional tendencies, etc. During use, creators can input instructions for content creation and editing, such as "Please write an article about environmental protection." Through the content creation system constructed by the second largest language model obtained through training, creators can more accurately control the content and style of the generated text, thereby improving creation efficiency and quality.

[0104] (2) Intelligent question-answering system scenario.

[0105] Reference Figure 2b , Figure 2b An optional schematic diagram of the text generation method provided in the embodiment of this application being applied in the context of an intelligent question-and-answer system. In fields such as intelligent customer service and online education, the question-and-answer system needs to accurately understand the user's questions and generate corresponding answers. During use, the user can enter question-and-answer instructions, such as "How do I clean my washing machine?" The intelligent question-and-answer system constructed by the trained second language model improves the quality and accuracy of the answers.

[0106] In addition, the method provided in the embodiments of the present application can be applied to different scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving and other scenarios.

[0107] Reference Figure 3 , Figure 3An optional flow chart of a text generation method provided in an embodiment of the present application is provided. The text generation method can be executed by a server or a terminal alone, or by a terminal and a server in cooperation. The text generation method includes but is not limited to the following steps 301 to 305.

[0108] Step 301: obtain an initial instruction and a constraint dimension text, call a first language model to add multiple constraint condition texts to the initial instruction based on the constraint dimension text, and output a first instruction.

[0109] The initial instruction is the initial text generation instruction during the training of the second language model in the text generation task. This is also known as a seed instruction. It can be a basic, uncomplicated instruction provided by the user or the system. Furthermore, the initial instruction can include some basic, necessary guidance information to guide the subsequent text generation process, such as "Please write an article about environmental protection."

[0110] Constraint dimension text is descriptive text used to further define and restrict the scope and content of text generation based on the initial instructions. It contains a series of specific requirements and constraints for the text generation process, guiding the first language model to generate sample text that meets specific needs and clearly indicating the specific requirements that should be followed in generation tasks or data processing tasks. There can be multiple types of constraint dimension text, each representing a corresponding constraint dimension. The text content contained in each constraint dimension is the constraint dimension text, so multiple constraint dimension texts also represent multiple constraint dimensions. Constraint condition text is a concrete rule or condition used to clearly indicate the specific requirements that should be followed in generation tasks or data processing tasks. Each type of constraint condition text represents text information under a constraint dimension. Therefore, the number of constraint condition texts contained in a first instruction also represents the number of constraints or constraint dimensions contained in the first instruction.

[0111] Exemplarily, word count constraint and tone constraint are two different constraint dimension texts, representing different constraint dimensions, wherein the word count constraint may include two specific constraints: length not exceeding 500 words and length of the article not exceeding 1000 words. The text generated based on these two specific constraints is the constraint text, or the length not exceeding 500 words and the length of the article not exceeding 1000 words can be called two specific constraint texts under the word count constraint dimension, but these two constraint texts belong to the same category. The style constraint may include two specific constraints: the formal style of the article and the humorous style. The text generated based on these two specific constraints is the constraint text, or the formal style of the article and the humorous style can be called two specific constraint texts under the style constraint dimension, but these two constraint texts belong to the same category. In addition, the word count constraint dimension, length not exceeding 500 words and length of the article not exceeding 1000 words, is a different category from the style constraint dimension, which is the formal style of the article and the humorous style.

[0112] The first large language model is a large pre-trained language model for text generation. It is built based on deep learning technology. By training on a large corpus, it learns the statistical laws and patterns of language, enabling it to generate natural and coherent text. For example, the first large language model is a large language model that has been pre-trained on large-scale data. These models have strong language understanding and generation capabilities and can handle a variety of complex text generation tasks.

[0113] It should be noted that the initial instruction is the starting point of the text generation task. By calling the first language model and adding multiple constraint condition texts based on the constraint dimension text, the initial instruction is expanded and complicated, generating a more precise and specific first instruction. This first instruction contains more details and constraints than the initial instruction, and therefore can guide the first language model to generate higher-quality sample text.

[0114] For example, if a seed instruction is "Please write an article about environmental protection," this instruction is used as the initial instruction. Assume that the two constraint dimension texts, "Article length" and "Article style," are currently selected. Based on these two constraint dimension texts and the initial instruction, the first language model is input for instruction generation. This generates a more precise and specific first instruction, such as "Please write an article about environmental protection, no longer than 500 words, and written in a formal, objective style." Here, "no longer than 500 words" and "written in a formal, objective style" are two different constraint texts. Ultimately, the article must meet the two specific constraints of "no longer than 500 words" and "formal, objective style."

[0115] Furthermore, the embodiment of the present application needs to construct a prompt text based on the initial instruction and the constraint dimension text, specifically including using the Prompt technology to allow the first language model to add specific constraint text to the initial instruction, thereby achieving the complexity of the instruction. Figure 4 , Figure 4 An optional schematic diagram of an embodiment of the present application for complicating instructions through prompt input may contain a prompt text such as "You are an expert in generating complex instructions. According to the description and example of the [constraint dimension] specified below, generate a clear, specific, and logical [specific constraint condition] for each constraint dimension that contains only one requirement, and then add these specific constraints to the [initial instruction] to generate the [complexed instruction]". Then, the prompt text also includes the specific initial instruction "seed_instruction" and the constraint dimension text of different constraint dimensions "constraint_dimensions". The complicated instruction is the first instruction. Through the input of the prompt text, the first language model will select multiple constraints from the constraint dimensions based on the initial instruction, and find the constraints under each dimension. Finally, the specific constraint text will be added to the initial instruction to obtain the complicated first instruction.

[0116] In addition, the prompt text can also limit the format of the output of the first language model, such as including "Finally, please strictly output the results according to [format]" in the prompt text, and attaching a specific format at the end. Not only that, other requirements for input and output can also be further restricted, which will not be repeated here.

[0117] In one possible implementation, some heuristic rules can be used to mine sample logs to obtain some modular initial instructions for training, further enhancing the model's ability to follow various types of complex instructions.

[0118] Step 302: Call the first large language model to delete at least one constraint text from the first instruction and output a second instruction.

[0119] After obtaining the first instruction, the embodiment of the present application also calls the first large language model to delete at least one constraint text from the first instruction, thereby completing the rewriting of the first instruction, and generating a rewritten second instruction, which has fewer constraint texts than the first instruction. Therefore, the second instruction is a simplified version of the first instruction. If there are multiple constraint texts contained in the first instruction, at least one constraint text can be deleted as needed. For example, if the first instruction contains five different types of constraint texts A, B, C, D and E, each of which is different, you can choose to delete 1, 2, 3 or even 4 types of constraint texts, such as deleting the two constraint texts A and B, to obtain a second instruction containing only the three constraint texts C, D and E.

[0120] It should be noted that calling the first large language model to delete at least one constraint text from the first instruction, rather than manually deleting it from the first instruction, can quickly generate multiple instructions at different levels. This is especially important when a large amount of data is required for model training or testing, thereby improving the automation and efficiency of training. Letting the same language model be responsible for adding and deleting constraints can ensure the consistency of the entire process, reduce the deviation that may be introduced by human intervention, and ensure the standardization of instruction modifications. Using the same large language model for the entire process (including adding and deleting constraints, generating text, etc.) can better control the experimental conditions, so that the quality differences of the generated text are more attributable to changes in constraints.

[0121] For example, the first instruction contains multiple types of constraint text. Therefore, when a certain number of constraint texts are deleted, different constraint dimensions, that is, different types of constraint texts, can be selectively deleted to obtain different second instructions. For example, if the first instruction contains constraint texts under five different constraint dimensions, A, B, C, D, and E, each of different types, then when choosing to delete one constraint text, any one of A, B, C, D, and E can be deleted, thereby obtaining a second instruction consisting of the remaining four constraint texts.

[0122] Furthermore, the embodiment of the present application needs to construct a prompt text of the rewriting process based on the first instruction, specifically including using the Prompt technology to allow the first language model to delete at least one constraint text from the first instruction, thereby simplifying the instruction. Figure 5 , Figure 5An optional schematic diagram of instruction rewriting through prompt input provided in an embodiment of the present application, in which the rewriting prompt text that may be contained is "You are an instruction rewriter, please remove one [constraint] from the [task instruction], do not make other additional changes, and obtain the [rewritten task instruction]", "task instruction" is the first instruction, and in addition, "task instruction" can also be other instructions that need to be rewritten. "Remove one [constraint]" means randomly removing the constraint text under any constraint condition, and in addition, you can specify which constraint text to remove. Not only that, the rewriting prompt text also includes the specific task instruction "instruction", the constraint dimension "constrint" contained in the task instruction, and the rewritten task instruction "instruct_rewrite", which is the second instruction. Since a constraint text is selected under each constraint dimension in this application and added to the initial instruction, a certain constraint dimension here represents the constraint text under the corresponding dimension. By inputting the rewriting prompt text, the first language model will delete at least one constraint text based on the task instruction, thereby simplifying the instruction and obtaining the simplified second instruction.

[0123] In addition, the rewritten prompt text may also limit the format of the output of the first language model, which will not be described in detail here.

[0124] Step 303: Call the first large language model to generate text based on the first instruction and the second instruction, and output sample texts corresponding to the first instruction and the second instruction respectively.

[0125] After obtaining the first instruction, the embodiment of the present application can call the first large language model, use the first instruction as input, and generate the corresponding sample text. Since the first instruction contains more constraints, it is expected that the generated text will be more accurate, rich, and meet the specified requirements, showing higher quality. Similarly, after obtaining the second instruction, the embodiment of the present application can also call the first large language model, but this time use the second instruction as input. Since the second instruction has fewer constraints, the generated sample text may be slightly inferior to the text generated based on the first instruction in some aspects (such as accuracy, relevance, etc.), resulting in slightly poorer quality of the generated sample text.

[0126] It is understandable that there can be multiple second instructions, and the multiple second instructions are instructions after deleting different types of constraint condition texts. Therefore, each second instruction can generate a corresponding sample text.

[0127] Step 304: Combine the sample texts to obtain a comparison text set, and train the second largest language model based on the comparison text set.

[0128] The comparison text set contains multiple sample texts, providing sample data with significant quality differences for training the second language model. Compared to the first language model, the second language model focuses on further training and optimization for specific tasks or scenarios. For example, the second language model is an SFT model, which requires subsequent fine-tuning or enhancement using specific datasets and training strategies.

[0129] It should be noted that by conducting comparative learning on different sample texts in the comparison text set constructed in this application, the second largest language model can identify the impact of different constraints on the quality of text generation, and learn how to generate high-quality text under different constraints, which helps to improve the second largest language model's ability to follow instructions and enable it to generate higher quality target text when subsequently generating text based on target instructions.

[0130] Specifically, when the second-largest language model is exposed to sample text corresponding to the first instruction, it can learn how to generate high-quality text based on complex and precise instructions. At the same time, when the second-largest language model is exposed to sample text of slightly lower quality corresponding to the second instruction, it can learn how to maintain a certain text quality when the constraints are reduced, or at least avoid generating text of extremely poor quality. Through training, the second-largest language model can improve its ability to follow instructions and, when subsequently generating text based on the target instruction, can more effectively generate high-quality target text.

[0131] Step 305: Obtain the target instruction, call the trained second language model to generate text based on the target instruction, and output the target text.

[0132] The target instruction is the text generation instruction for the second language model in the text generation task. It can be a user- or system-provided instruction for the current text generation task. Unlike the initial instruction, first instruction, or second instruction, the target instruction does not require specific complexity to achieve high-quality text generation. Furthermore, the target instruction can be relatively simple and clear, only needing to include basic requirements, such as "Please write an article about environmental protection."

[0133] The target text is the final text generated by the second largest language model according to the target instruction. It is the response and output of the second largest language model to the target instruction, and is intended to meet the requirements put forward by the user. The quality of the target text depends on the second largest language model's understanding and ability to follow the target instruction, as well as the text generation skills learned by the model during the training process. The embodiment of the present application trains the second largest language model to better follow the target instruction and learn a higher quality text generation method, so the target text ultimately generated is of higher quality.

[0134] In summary, the text generation method of the embodiment of the present application obtains an initial instruction and a constraint dimension text, calls the first language model to add multiple constraint texts to the initial instruction based on the constraint dimension text, thereby outputting a more complex first instruction, then calls the first language model to delete at least one constraint text from the first instruction, outputs a second instruction, calls the first language model to generate text based on the first instruction and the second instruction respectively, and outputs sample texts corresponding to the first instruction and the second instruction respectively. Since the complicated first instruction is more accurate and specific, the quality of the generated sample text is better, and since the number of constraint text types contained in the second instruction is less than that contained in the first instruction, the quality of the generated sample text is slightly worse. Therefore, by combining the sample texts, a comparative text set consisting of sample texts with different qualities can be obtained. Subsequently, the second language model can be trained based on the comparative text set, so that the second language model learns a better quality text generation method, thereby improving the second language model's ability to follow the instruction. Subsequently, the target instruction is obtained, and when the trained second language model is called to generate text based on the target instruction, the quality of the generated target text can be effectively improved.

[0135] Above, the details of the overall process of the text generation method in the embodiment of the present application are introduced through steps 301 to 305. Below, the contents further included in step 301 are described in detail:

[0136] In one possible implementation, the constraint dimension text has multiple levels, and thus selection can be made from the multiple levels of constraint dimension text. Specifically, the first dimension text can be determined from the constraint dimension texts at the first level, and the second dimension text can be determined from the constraint dimension texts at the Nth level associated with the first dimension text, where N is an integer and N ≥ 2. The first language model is then invoked to add multiple constraint condition texts to the initial instruction based on the second dimension text, and the first instruction is output.

[0137] It should be noted that the constraint dimension text is organized into multiple levels, each level contains a series of constraint dimension texts, these levels include the first level and other levels after the first level, among which the first level is the highest level, that is, it represents the top-level constraint dimension, which can contain the most extensive or basic constraint dimension text. For example, the first level can include the second level, and the second level can include the third level or more. It should be noted that some constraint dimensions of the first level may not contain the constraint dimensions of the next level, and some constraint dimensions of the first level may contain the constraint dimensions of the next level, that is, the second level. Furthermore, higher levels can be set above the first level as needed, and no specific restrictions are made here.

[0138] Determining the first dimension text from the various constraint dimension texts at the first level means that when generating the first instruction, one or more specific constraint dimension texts at the first level must be selected as the starting point. For example, suppose you want to generate an article about travel. The first level of constraint dimension texts might include different constraint dimension texts such as "travel destination," "travel method," and "travel time." If "travel destination" is selected as the first dimension text, then the subsequent constraint condition texts and the generated text will all revolve around this specific destination.

[0139] Then, after determining the first dimensional text, the first dimensional text serves as the constraint dimensional text of the first level, and multiple levels below it may be associated with it. The second dimensional text needs to be determined from the constraint dimensional text of the Nth level associated with the first dimensional text, where N is an integer, N≥2, that is, the second dimensional text needs to be determined from the constraint dimensional text of the second level or a higher number of levels greater than 2. Subsequently, the first language model can be called to add multiple constraint condition texts to the initial instruction based on the second dimensional text to output the first instruction.

[0140] For example, to generate an article about environmental protection, the length constraint, language constraint, and style constraint of the article all belong to the first-level constraint dimension text. After selecting the style constraint as the first-level constraint dimension text, the second-level constraint dimension text can be determined in the next-level constraint dimension text associated with the style constraint. For example, the style constraint can be associated with three second-level constraint dimension texts: tone constraint, celebrity imitation constraint, and writing style constraint. After selecting the tone constraint as the second-level constraint dimension text, the first language model can be called to add multiple constraint condition texts to the initial instruction based on the second-dimensional text to output the first instruction.

[0141] Furthermore, when the constraint dimension text of the second level is also associated with the constraint dimension text of the next level, it is necessary to continue to determine the second dimension text of the next level. For example, the tone constraint can be associated with two constraint dimension texts of the third level, the formal constraint and the humor constraint, so one of the formal constraint and the humor constraint is determined to be the second dimension text. Therefore, the second dimension text in the embodiment of the present application is the constraint dimension text of the last level associated with the multi-level constraint dimension text, which can make the instructions more precise and specific, and ensure that the constraints used are the most direct and relevant, thereby avoiding redundancy and potential conflicts.

[0142] For example, a complete constraint system is pre-built in the embodiment of the present application, with a total of ten constraint dimensions, including quantity constraint, negation constraint, length constraint, layout constraint, language constraint, style constraint, theme constraint, format constraint, content constraint and style constraint. Figure 6 , Figure 6 This is an optional schematic diagram of a constraint system provided in an embodiment of the present application. The multi-level constraint dimension text in the figure is as follows:

[0143] (1) Quantity constraints are associated with multiple second-level constraint dimension texts, such as number / frequency constraints and range constraints, while range constraints are associated with multiple third-level constraint dimension texts, such as greater than / greater than or equal to, less than / less than or equal to, and between;

[0144] (2) Negative constraint is the first level constraint dimension text, and there is no other level associated with it;

[0145] (3) Length constraints are associated with multiple second-level constraint dimensions such as the number of words, the number of paragraphs, and the number of sentences;

[0146] (4) The typesetting constraint is associated with multiple second-level constraint dimension texts such as tables and lists, and the table constraint is associated with multiple third-level constraint dimension texts such as ordered lists and unordered lists;

[0147] (5) Language constraints are associated with multiple second-level constraint dimension texts such as English, Chinese, Japanese, and English. Chinese constraints are associated with multiple second-level constraint dimension texts such as Simplified Chinese and Traditional Chinese.

[0148] (6) Style constraints are associated with multiple second-level constraint dimension texts such as highlight / bold, indent, and color;

[0149] (7) The subject constraint is the first level constraint dimension text, and there is no other level associated with it;

[0150] (8) The format constraint is associated with a second-level constraint dimension text, namely, format specification. The format specification is associated with multiple third-level constraint dimension texts, such as yaml format, json format, html format, xml format, markdown format, and csv format;

[0151] (9) Content constraints are associated with multiple second-level constraint dimension texts such as keywords, symbols, entities, time, structure, and restricted content; symbol constraints are associated with multiple third-level constraint dimension texts such as punctuation and emoticons; structure constraints are associated with multiple third-level constraint dimension texts such as N parts, introduction, title, postscript, and conclusion; restricted content is associated with multiple third-level constraint dimension texts such as inclusion and appearance position;

[0152] (10) Style constraints are associated with multiple second-level constraint dimension texts, such as tone, imitation of celebrities, and writing style. Tone constraints are associated with multiple third-level constraint dimension texts, such as formality and humor.

[0153] The constraint system in the embodiment of the present application is only an example. In practice, other constraint systems can be selected as needed to complicate the instructions, and the embodiment of the present application does not impose any specific restrictions on this.

[0154] In one possible implementation, the constraint dimension texts at the first level can have corresponding semantic tags. By identifying the semantic tags in the initial instruction, conditional screening can be further performed based on these semantic tags. Specifically, semantic identification can be performed on the initial instruction to obtain the first semantic tag of the initial instruction; the second semantic tags preset for each constraint dimension text at the first level are obtained and matched with the first semantic tags; when the second semantic tags match the first semantic tags, the constraint dimension text is determined to be the first dimension text.

[0155] Among them, semantic recognition is equivalent to classifying the initial instructions. The first semantic label is the label obtained directly from the initial instruction through semantic recognition. It represents the main content or intention of the initial instruction and is a refinement and annotation of the core semantics of the initial instruction. The second semantic label is a label pre-set in the constraint dimension text of the first level. It reflects the subject, field or attribute of each constraint dimension text and is a general description of the content or characteristics of these texts. Therefore, the first semantic label needs to be identified, and the second semantic label is pre-set in the constraint dimension text.

[0156] Exemplarily, there are many ways to perform semantic recognition on the initial instruction and obtain the first semantic label of the initial instruction. For example, a rule-based method can be used to match the text in the initial instruction with a preset list of keywords or phrases to determine the first semantic label; the initial instruction can also be regarded as a collection of words, without considering the word order, and the first semantic label can be predicted by statistical features such as word frequency; the word frequency-inverse document frequency algorithm can also be used to calculate the importance of each word, and then determine the semantic label of the instruction; the initial instruction can also be input into a preset convolutional neural network (CNN), recurrent neural network (RNN) or other neural network model for semantic recognition through deep learning methods, so as to obtain the required first semantic label. In actual applications, which method can be selected for semantic recognition of the initial instruction depends on the specific application scenario, data volume, computing resources and accuracy requirements, and no specific restrictions are made here.

[0157] It should be noted that after matching the second semantic tag with the first semantic tag, a matching result between the two semantic tags can be obtained. When the second semantic tag is consistent with the first semantic tag, the corresponding constraint dimension text is selected as the first dimension text and enters the next processing or analysis process, making the screening and classification of text more accurate and efficient, and helping to quickly locate information of interest from a large amount of data. For example, if the initial instruction is to generate an article about environmental protection, the first semantic tag obtained by identification is "environmental protection", then among the many second semantic tags, it is necessary to find those related to environmental protection, such as the constraint dimension text of certain content, which contains content related to climate change, energy, etc., then the associated first-level constraint dimension text will contain the second semantic tags of "climate change", "renewable energy" or even "environmental protection", and then the corresponding first-level constraint dimension text will be determined as the first-dimensional text.

[0158] It's understandable that a single constraint dimension text can contain multiple types of second semantic tags, depending on the specific content of the constraint dimension texts at other levels of its association. This is not a specific limitation here. If a second semantic tag is inconsistent with the first semantic tag, a search is performed again in the constraint dimension texts at the first level to find a second semantic tag that matches the first semantic tag.

[0159] In one possible implementation, different constraint dimension texts can be assigned corresponding constraint weights. Based on these weights, the initial instruction can be complicated to control the complexity of the added constraint dimension text. Specifically, the constraint weights corresponding to each constraint dimension text can be determined, and a third instruction can be constructed based on the initial instruction, the constraint dimension text, and the constraint weights. The third instruction prompts the user to add multiple constraint condition texts to the initial instruction. The third instruction is then input into the first large language model for text generation, which then outputs the first instruction.

[0160] In an embodiment of the present application, before starting to complicate the initial instructions, a constraint weight is first assigned to each constraint dimension text. The constraint weight represents the importance and complexity of the constraint dimension text in the final instruction. The higher the constraint weight, the more complex the constraint condition text finally added will be.

[0161] Next, with the initial instruction and the weights corresponding to each constraint dimension text determined, a third instruction can be constructed based on this. The third instruction is a guideline that provides instructions on how to complicate the initial instruction based on the constraint dimension texts with different constraint weights. This constructed third instruction is then input into the first large language model. Based on the third instruction's prompts and the constraint weights of the constraint dimension texts, the first language model generates a new instruction, the first instruction, thus complicating the initial instruction.

[0162] Furthermore, the embodiment of the present application needs to construct prompt text based on the initial instruction and the constraint dimension text with different constraint weights, specifically including using Prompt technology to let the first language model add specific constraint condition text to the initial instruction, thereby realizing the complexity of the instruction. Figure 7 , Figure 7 An optional schematic diagram of an embodiment of the present application for complicating instructions by using constraint weights in prompt input may contain prompt text such as "You are an expert in generating complex instructions. According to the description and examples of the [constraint dimension] and [constraint weight] specified below, generate clear, specific, and logical [specific constraints] for each constraint dimension that only contains one requirement. Then add these specific constraints to the [seed instruction] to generate the [complexed instruction]. Finally, please output the result strictly in accordance with the [format]". Then, the prompt text also includes the specific initial instruction "seed_instruction", as well as the constraint dimension texts of different constraint dimensions "constraint_dimensions" and the corresponding constraint weights "weight". Through the input of the prompt text, the first language model will select multiple constraint dimension texts of different weights on the basis of the initial instruction, and find the constraints under each dimension. Finally, the specific constraint text will be added to the initial instruction to obtain the first complicated instruction.

[0163] For example, there are multiple ways to determine the constraint weights corresponding to each constraint dimension text. For example, each constraint dimension text can be pre-configured with corresponding weights when it is established, so these weights can be directly used as the corresponding constraint weights. In addition, the constraint weights corresponding to each constraint dimension text can also be generated through a weight determination model. The weight allocation model can be any neural network model, such as a regression model. The following describes in detail how to obtain the constraint weights corresponding to each constraint dimension text through the weight determination model:

[0164] In one possible implementation, the constraint weights corresponding to each constraint dimension text are obtained through a weight determination model. Specifically, the initial instruction and the constraint dimension text are encoded separately to obtain a first embedding vector for the initial instruction and a second embedding vector for each constraint dimension text. The first embedding vector is concatenated with each second embedding vector to obtain multiple target embedding vectors. The weight determination model is then invoked to perform regression processing on each target embedding vector to obtain the constraint weights corresponding to each constraint dimension text. Furthermore, the training process can also jointly train the second language model and the weight determination model based on the comparison text set.

[0165] The weight determination model is a machine learning model. Jointly training the weight determination model with the second largest language model enables effective feature extraction and fusion, optimizing the entire denoising and text generation process and improving overall performance. This joint training approach helps ensure coordination and consistency between the different components, making the model more adaptable to actual task requirements.

[0166] Before determining the constraint weights corresponding to each constraint dimension text, it is first necessary to construct the input data of the weight determination model. The embodiment of the present application needs to encode the initial instruction and the constraint dimension text respectively, and convert them into embedding vectors to convert the text data into a numerical form that the model can process, and obtain the first embedding vector of the initial instruction and the second embedding vector of each constraint dimension text. The first embedding vector and the second embedding vector can capture the semantic information of the text, so that similar texts are closer in the vector space. Then, the first embedding vector of the initial instruction is spliced ​​with the second embedding vector of each constraint dimension text respectively to fuse the semantic information of the two. The splicing operation includes connecting the two vectors to form a new, longer vector. This new vector contains both the semantic information of the initial instruction and the semantic information of the corresponding constraint dimension text. Finally, by calling the weight determination model to regress the spliced ​​target embedding vector, the model can learn how to determine the constraint weight based on the semantic information of the initial instruction and the constraint dimension text.

[0167] It should be noted that by concatenating the embedding vectors of the initial instructions and the constraint dimension text, and combining them with regression processing, the weight determination model can more accurately capture the relationship between the two, thereby obtaining more accurate constraint weights. This helps to accurately add the constraint dimension text according to the constraint weights in the subsequent text generation process, thereby complicating the initial instructions. During the training process, the goal of the weight determination model is to minimize the loss function, which measures the difference between the weights predicted by the model and the true or ideal weights. By adjusting the model parameters (including the constraint weights corresponding to each constraint dimension text), the weight determination model minimizes this loss, thereby improving its prediction accuracy. As training progresses, the model will learn how to assign appropriate weights to different input features. Therefore, for a given target embedding vector, the weight determination model can automatically determine the proportion that each constraint dimension text should occupy in the initial instruction.

[0168] Reference Figure 8 , Figure 8 An optional schematic diagram of the process of outputting the first instruction after assigning constraint weights to different constraint dimension texts provided in the embodiment of the present application. Taking two constraint dimension texts as an example, after encoding the first embedding vector of the initial instruction and the second embedding vectors (second embedding vector 1 and second embedding vector 2) of the two constraint dimension texts (constraint dimension text 1 and constraint dimension text 2), the first embedding vector is spliced ​​with the two second embedding vectors respectively to obtain two target embedding vectors (target embedding vector 1 and target embedding vector 2), and the two target embedding vectors are input into the weight determination model for regression processing respectively to obtain the constraint weight 1 corresponding to the constraint dimension text 1 and the constraint weight 2 corresponding to the constraint dimension text 2. Subsequently, based on the initial instruction, different constraint dimension texts and the corresponding constraint weights, the required third instruction can be constructed, the third instruction is input into the first large language model for text generation, and the final first instruction is output.

[0169] The above describes the details of the further contents included in step 301. The following describes in detail the further contents included after step 303:

[0170] In one possible implementation, after calling the first large language model to generate text based on the first instruction and the second instruction respectively, and outputting the sample text corresponding to the first instruction and the second instruction respectively, the output sample text needs to be verified. Specifically, the first large language model can be called to verify the sample text, and the verification result and the explanation text can be output; wherein the verification result is used to indicate the integrity of the sample text, and the explanation text is used to explain the verification result.

[0171] It should be noted that the first and second instructions are complex because they contain multiple constraint texts. Therefore, the answer initially generated by the first language model may not fully cover or correctly understand these requirements, that is, the initially generated sample text is imperfect. Based on this, the sample text needs to be verified.

[0172] Specifically, the embodiment of the present application calls the first language model to verify the sample text, and outputs the verification result and the explanation text. Among them, the verification result is a brief indication for telling whether the sample text meets all the constraint texts, that is, how complete it is. For example, the verification result can be binary (for example, pass / fail, complete / incomplete), or quantitative (for example, the percentage of constraint texts that meet), or graded (for example, excellent / good / general / poor), which directly reflects the extent to which the sample text meets the requirements of the constraint text. The explanation text is a detailed explanation or description of the verification result, providing detailed information about why the sample text meets or does not meet the constraint text. Further, the explanation text can include pointing out missing information, identifying errors or misunderstandings, and suggestions for improvement.

[0173] In one possible implementation, after calling the first language model to verify the first instruction and outputting the verification result and the explanatory text, when the verification result indicates that the sample text is incomplete, the sample text needs to be improved and refined. Specifically, the first language model can be called to regenerate the text based on the first instruction and the second instruction respectively until the complete sample text is output.

[0174] Furthermore, when the verification result indicates that the sample text is incomplete, it means that the sample text is not perfect and needs to be improved and refined. At this time, the first language model can be called to regenerate the text based on the first instruction and the second instruction respectively until the complete sample text is output. Therefore, through the refinement process, it can be ensured that the sample text accurately reflects all the requirements in the instruction to avoid omissions or misunderstandings.

[0175] Furthermore, the embodiment of the present application needs to construct a prompt text of the refinement process based on the sample instruction and the input first instruction or second instruction, specifically including using the Prompt technology to let the first language model refine the sample text. Figure 9 , Figure 9An optional schematic diagram of sample text refinement through prompt input provided in an embodiment of the present application, wherein the refinement prompt text that may be contained in the diagram is "You are an answer refinement expert. Please judge whether the following [sample text] fully meets all the requirements of the [task instruction] and give a reasonable explanation. Then, based on the given judgment and explanation, refine the [sample text] again to generate a refined sample text to ensure that all the requirements of the [task instruction] are met." "Task instruction" is the first instruction or the second instruction, which refers to any instruction that contains constraint text. Not only that, the refinement prompt text also includes a specific task instruction "instruction" and a sample text "answer". Through the input of the refinement prompt text, the first language model will refine the sample text based on the task instruction, thereby obtaining a perfect sample text.

[0176] In addition, the refined prompt text may also limit the format of the output of the first language model, which will not be described in detail here.

[0177] The above describes the details of the steps following step 303. The following describes the details of the steps following step 304:

[0178] In one possible implementation, the first large language model outputs multiple types of second instructions. Each type of second instruction is obtained by deleting different amounts of constraint text. Therefore, multiple sample texts are generated based on these second instructions, thereby constructing the desired comparison document set. Specifically, each sample text is used as a benchmark text, and the benchmark text is combined with at least one remaining sample text to generate the comparison document set; wherein the number of constraint texts satisfied by the benchmark text is greater than the number of constraint texts satisfied by the remaining sample text.

[0179] It should be noted that in the embodiments of the present application, different amounts of constraint text are deleted, that is, different types of constraint text are deleted, thereby obtaining multiple different second instructions. For example, if a first instruction contains constraint text under five different constraint dimensions, each of different types, by deleting different amounts of constraint text, a second instruction containing four different constraint texts, a second instruction containing three different constraint texts, a second instruction containing two different constraint texts, and so on, can be obtained. The sample texts generated by these second instructions containing different amounts of constraint texts also have different qualities, and the sample texts generated by the second instructions containing a large number of constraint texts are of higher quality, while the sample texts generated by the second instructions containing a small number of constraint texts are of lower quality.

[0180] Next, one of the sample texts can be selected as a baseline text, and the baseline text can be combined with at least one of the remaining sample texts to form a comparison document set. The baseline text is used as a standard or reference for comparison with the other sample texts, and the baseline text satisfies a greater number of constraint texts than the remaining sample texts. In other words, the baseline text is generated by adding more constraints to the initial instructions, making it more accurate, specific, and of higher quality. Therefore, in the resulting comparison document set, by comparing the baseline text with sample texts that meet fewer constraints, the impact of different constraints on text generation quality can be clearly demonstrated.

[0181] In one possible implementation, the comparison document set includes a benchmark text and one of the remaining sample texts. Based on this, multiple local sample pairs can be constructed as the comparison text set, and the second largest language model can be trained. Specifically, a first reward score for the second largest language model can be determined based on the benchmark text, a second reward score for the second largest language model can be determined based on the remaining sample text, and a first ratio between the first reward score and the second reward score can be determined; a third reward score for the third largest language model can be determined based on the benchmark text, a fourth reward score for the third largest language model can be determined based on the remaining sample text, and a second ratio between the third reward score and the fourth reward score can be determined; a first loss can be determined based on the first ratio and the second ratio, and the second largest language model can be trained based on the first loss.

[0182] When the comparison document set includes a reference text and one of the remaining sample texts, multiple local sample pairs can be constructed as the comparison text set. The reference text is a better sample, also known as a positive example, and one of the remaining sample texts is a worse sample, also known as a negative example. The reference text can be used first for SFT, and then the reference text and the remaining sample text are used to construct a comparison text set for comparative training.

[0183] For example, if the first instruction contains five different constraint texts, then the sample text generated by the first instruction is r1, the sample text generated by the second instruction containing four constraint texts is r2, the sample text generated by the second instruction containing three constraint texts is r3, the sample text generated by the second instruction containing two constraint texts is r4, and the sample text generated by the second instruction containing one constraint text is r5. Sorted by quality, they are r1, r2, r3, r4, and r5 respectively. Then any sample text except r5 can be selected as the benchmark text, and together with another sample text with poor quality in the sorting, a local comparison sample is formed. This local comparison sample is the comparison text set. For example, [r1, r2], [r1, r3], [r1, r4], and [r1, r5] are multiple comparison text sets constructed with r1 as the benchmark text. In [r1, r2], r1 is the better sample, and r2 is the worse sample. The others are not repeated here.

[0184] Among them, the third largest language model can be the same model as the second largest language model, and the model parameters are frozen and do not participate in training. Its role is to ensure that the distribution difference between the second largest language model and the third largest language model is not too large during the learning process.

[0185] After obtaining multiple comparison text sets, the second largest language model can be compared and learned based on the comparison text sets. Figure 10 , Figure 10 An optional schematic diagram of comparative learning of a second language model provided in an embodiment of the present application. During the training process, a reward score is introduced to evaluate the closeness of the text generated by the second language model to the benchmark text or sample text. The reward score can be determined based on a variety of factors, such as the semantic similarity, grammatical correctness, and fluency of the generated text and the benchmark text. Since the benchmark text is closer to the text generated by the model, it will receive a higher reward score. Specifically, the benchmark text and the corresponding first instruction can be input into the reward model to obtain a reward score for the state-action pair.

[0186] Therefore, a first reward score for the second-largest language model can be determined based on the benchmark text, a second reward score for the second-largest language model can be determined based on the remaining sample text, and a first ratio between the first reward score and the second reward score can be determined. The first ratio is the proportional relationship between the first reward score and the second reward score, and this ratio reflects the performance difference between the model and the high-quality benchmark text and the low-quality sample text when generating text. Next, a third-largest language model is introduced and compared with the second-largest language model on the same task. A third reward score for the third-largest language model is determined based on the benchmark text, and a fourth reward score for the third-largest language model is determined based on the remaining sample text. A second ratio between the third reward score and the fourth reward score is determined. The second ratio is the proportional relationship between the third reward score and the fourth reward score, and can be used for comparison or reference with the performance of the second-largest language model. Finally, a first loss is determined based on the first and second ratios. Since the first loss measures the closeness of the second-largest language model's generated text to the high-quality benchmark text and the difference from other sample text, by minimizing the first loss, the second-largest language model can be trained to be more inclined to generate high-quality text similar to the benchmark text.

[0187] Furthermore, in the embodiment of the present application, after selecting the benchmark text, a comparison text set can be constructed by obtaining sample files output by other large language models. Specifically, the second instruction containing a poor number of constraint texts can be input into other large language models for text generation to obtain corresponding sample texts, which are also of poor quality and worse than the benchmark text. For example, if the sample text r1 generated according to the first instruction containing five constraint texts is used as the benchmark text, the second instruction containing four constraint texts can be input into the first large language model, and the generated sample text is r21. The same second instruction containing four constraint texts is then input into other large language models, and the generated sample text is r22. In this way, local sample pairs [r1, r21] and [r1, r22] can be constructed, and the second large language model can be trained.

[0188] In one possible implementation, the comparison document set may further include a benchmark text and a plurality of remaining sample texts, based on which a local sorted sample may be constructed as the comparison text set, and the second largest language model may be trained. Specifically, a first reward score for the second largest language model may be determined based on the benchmark text, a second reward score for the second largest language model may be determined based on each of the remaining sample texts, and a third ratio between the first reward score and the sum of the plurality of second reward scores may be determined; a third reward score for the third largest language model may be determined based on the benchmark text, a fourth reward score for the third largest language model may be determined based on each of the remaining sample texts, and a fourth ratio between the third reward score and the sum of the plurality of fourth reward scores may be determined; a second loss may be determined based on the third ratio and the fourth ratio, and the second largest language model may be trained based on the second loss.

[0189] When the comparison document set includes a benchmark text and multiple remaining sample texts, local sorting samples can be constructed based on this as the comparison text set. Among them, the benchmark text is a better sample, also known as a positive example, and the remaining other samples are worse samples, also known as negative examples. For example, referring to Figure 11 , Figure 11 An optional schematic diagram of constructing a ranking list for training the second language model provided in an embodiment of the present application, if the first instruction contains five different constraint texts ①+②+③+④+⑤, then the sample text generated by the first instruction is r1, and the sample text generated by the second instruction containing the constraint text ②+③+④+⑤ is r2, the sample text generated by the second instruction containing the constraint text ③+④+⑤ is r3, the sample text generated by the second instruction containing the constraint text ④+⑤ is r4, and the sample text generated by the second instruction containing the constraint text ④+⑤ is r5. The best sample text is r1, and the sample texts are ranked by quality, namely sample text r1, sample text r2, sample text r3, sample text r4, and sample text r5, so a ranking list (rank list) can be generated. The rank list is [r1, r2, r3, r4, r5] (i.e., the comparison document set), where sample text r1 is the better sample, sample text r2 is the worse1 sample, sample text r3 is the worse2 sample, sample text r4 is the worse3 sample, and sample text r5 is the worse4 sample. Worse1 is better than worse2, worse2 is better than worse3, and worse3 is better than worse4. In addition, other sample texts besides sample text r1 can be used as benchmark texts to construct other rank lists, such as [r2, r3, r4, r5], [r3, r4, r5], and [r4, r5].

[0190] It is understandable that the Figure 11In the example shown, the sample text can also be generated based on the second instruction containing the constraint text ①+②③+④, or generated based on the second instruction containing the constraint text ①+③+④, and so on.

[0191] In one possible implementation, again referring to Figure 11 The training of the second language model can be multiple rounds. In the first round, SFT is first performed based on the sample text r1, and then rank list [r1, r2, r3, r4, r5], rank list [r2, r3, r4, r5], rank list [r3, r4, r5], and rank list [r4, r5] are respectively constructed for comparative learning; in the next round, a constraint is deleted from the first instruction. For example, after removing the constraint condition text ①, SFT is first performed based on the sample text r2, and then rank list [r2, r3, r4, r5], rank list [r3, r4, r5], and rank list [r4, r5] are respectively constructed for comparative learning, and so on.

[0192] After obtaining the rank list, you can perform comparative learning on the second largest language model based on the rank list. Figure 12 , Figure 12 Another optional schematic diagram of comparative learning of the second language model provided in an embodiment of the present application. During the training process, in order to evaluate the degree of closeness of the text generated by the second language model to the benchmark text or sample text, a reward score is introduced. The reward score can be determined based on multiple factors, such as the semantic similarity, grammatical correctness, fluency, etc. between the generated text and the benchmark text. Since the benchmark text is closer to the text generated by the model, a higher reward score will be obtained.

[0193] Therefore, the first reward score of the second largest language model can be determined based on the benchmark text, the second reward score of the second largest language model can be determined based on the remaining sample texts, and the third ratio between the first reward score and the sum of multiple second reward scores can be determined; the third reward score of the third largest language model can be determined based on the benchmark text, the fourth reward score of the third largest language model can be determined based on the remaining sample texts, and the fourth ratio between the third reward score and the sum of multiple fourth reward scores can be determined; the second loss can be determined according to the third ratio and the fourth ratio, and the second largest language model can be trained based on the second loss.

[0194] Similarly, a first reward score for the second-largest language model is determined based on the baseline text, a second reward score for the second-largest language model is determined based on each of the remaining sample texts, and a third ratio is determined between the first reward score and the sum of the multiple second reward scores. The third ratio is the proportional relationship between the first reward score and the sum of the second reward scores. This ratio reflects the performance difference between the model and the high-quality baseline text and multiple low-quality sample texts when generating text, and is more accurate than the first ratio. Next, a third-largest language model is introduced and compared with the second-largest language model on the same task. A third reward score for the third-largest language model is determined based on the baseline text, and a fourth reward score for the third-largest language model is determined based on each of the remaining sample texts. A fourth ratio is determined between the third reward score and the sum of the multiple fourth reward scores. The fourth ratio is the proportional relationship between the third reward score and the sum of the multiple fourth reward scores, which can be used for comparison with the performance of the second-largest language model or as a reference. Finally, the second loss is determined based on the third ratio and the fourth ratio. Since the second loss measures the closeness of the second-largest language model to the high-quality benchmark text when generating text, as well as the difference from multiple other sample texts, by minimizing the second loss, the second-largest language model can be trained to make it more inclined to generate high-quality texts that are ranked higher.

[0195] The following describes the detailed process of the text generation method in the embodiment of the present application. Figure 13 , Figure 13A kind of optional schematic diagram of the complete training process of the text generation method provided by the embodiment of the present application. First, by obtaining the initial instruction and obtaining the pre-established constraint system, the constraint system contains multiple constraint dimension texts, such as quantity constraint, negation constraint, length constraint and typesetting constraint, etc., then calling the first large language model to add multiple constraint condition texts to the initial instruction based on the constraint dimension text, thereby outputting a more complex first instruction, completing the complexity of the instruction. Then calling the first large language model again to delete at least one constraint condition text from the first instruction, outputting the second instruction, calling the first large language model to generate text based on the first instruction and the second instruction respectively, outputting the sample text corresponding to the first instruction and the second instruction respectively, during which the generated sample text can be refined so that the sample text meets the requirements of all constraint condition texts in the instruction. Since the first instruction after complication is more accurate and specific, the quality of the generated sample text is better, and since the type of constraint condition text contained in the second instruction is less than the type of constraint condition text contained in the first instruction, the quality of the generated sample text is slightly worse, therefore, by combining the sample texts, a comparison text set consisting of sample texts with different qualities can be obtained. Then, the second language model can be trained for comparative learning based on the comparison text set, so that the second language model can learn a better quality text generation method, thereby improving the second language model's ability to follow instructions. Subsequently, when the target instruction is obtained and the trained second language model is called to generate text based on the target instruction, the quality of the generated target text can be effectively improved.

[0196] Furthermore, the second language model can learn how each constraint in an instruction will change the answer, allowing it to truly understand the meaning of each constraint, thereby strengthening the model's learning of the input-output mapping relationship, and ultimately improving the second language model's constraint pass rate for complex instructions and its generalization ability for out-of-domain tasks.

[0197] Reference Figure 14 , Figure 14 An optional flow chart of a model training method provided in an embodiment of the present application. The model training method can be executed by a server, or by a terminal and a server in cooperation. The model training method includes but is not limited to the following steps 1401 to 1404.

[0198] Step 1401: Obtain an initial instruction and constraint dimension text, invoke a first language model to add multiple constraint condition texts to the initial instruction based on the constraint dimension text, and output a first instruction;

[0199] Step 1402: Call the first large language model to delete at least one constraint text from the first instruction, and output a second instruction.

[0200] Step 1403: Call the first large language model to generate text based on the first instruction and the second instruction, and output sample text corresponding to the first instruction and the second instruction respectively;

[0201] Step 1404 : Combine the sample texts to obtain a comparison text set, and train the second largest language model based on the comparison text set.

[0202] It should be noted that steps 1401 to 1404 in the embodiment of the present application are similar to steps 301 to 304 in the above-mentioned text generation method. The training process can refer to the description in the previous embodiment and will not be repeated here.

[0203] In summary, the model training method in the embodiment of the present application obtains the initial instruction and the constraint dimension text, calls the first language model to add multiple constraint texts to the initial instruction based on the constraint dimension text, thereby outputting a more complex first instruction, and then calls the first language model to delete at least one constraint text from the first instruction, outputting the second instruction, calling the first language model to generate text based on the first instruction and the second instruction respectively, and outputting the sample text corresponding to the first instruction and the second instruction respectively. Since the complicated first instruction is more accurate and specific, the quality of the generated sample text is better, and since the number of constraint text types contained in the second instruction is less than that contained in the first instruction, the quality of the generated sample text is slightly worse. Therefore, by combining the sample texts, a comparative text set composed of sample texts with different quality can be obtained, and then the second language model can be trained based on the comparative text set, so that the second language model can learn a better quality text generation method, thereby improving the second language model's ability to follow the instructions.

[0204] It will be appreciated that, although the various steps in the above-mentioned various flow charts are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless clearly stated in the present embodiment, the execution of these steps does not have strict order restrictions, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the above-mentioned flow charts can include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0205] Reference Figure 15 , Figure 15 This is an optional structural diagram of a text generation device provided in an embodiment of the present application. The text generation device 1500 includes:

[0206] The first instruction generation module 1501 is configured to obtain an initial instruction and a constraint dimension text, invoke a first language model to add multiple constraint condition texts to the initial instruction based on the constraint dimension text, and output a first instruction;

[0207] A first instruction rewriting module 1502 is configured to call the first large language model to delete at least one constraint text from the first instruction and output a second instruction;

[0208] The first sample generation module 1503 is configured to call the first large language model to generate text based on the first instruction and the second instruction, and output sample texts corresponding to the first instruction and the second instruction respectively;

[0209] A first training module 1504 is configured to combine sample texts to obtain a comparison text set, and train a second language model based on the comparison text set;

[0210] The text generation module 1505 is used to obtain the target instruction, call the trained second language model to generate text based on the target instruction, and output the target text.

[0211] Furthermore, the first large language model outputs multiple types of second instructions, each of which is obtained by deleting different amounts of constraint text. The first training module 1504 is specifically configured to:

[0212] Taking each sample text as a reference text, combining the reference text with at least one remaining sample text to obtain a comparison document set;

[0213] The number of constraint condition texts satisfied by the benchmark text is greater than the number of constraint condition texts satisfied by the remaining sample texts.

[0214] Furthermore, the comparison document set includes the benchmark text and one of the remaining sample texts. The first training module 1504 is further configured to:

[0215] Determine a first reward score for the second largest language model based on the benchmark text, determine a second reward score for the second largest language model based on the remaining sample text, and determine a first ratio between the first reward score and the second reward score;

[0216] Determining a third reward score for the third largest language model based on the benchmark text, determining a fourth reward score for the third largest language model based on the remaining sample text, and determining a second ratio between the third reward score and the fourth reward score;

[0217] A first loss is determined according to the first ratio and the second ratio, and the second largest language model is trained based on the first loss.

[0218] Furthermore, the comparison document set includes a reference text and a plurality of remaining sample texts. The first training module 1504 is further configured to:

[0219] Determining a first reward score for the second largest language model based on the benchmark text, determining a second reward score for the second largest language model based on each of the remaining sample texts, and determining a third ratio between the first reward score and the sum of the plurality of second reward scores;

[0220] Determining a third reward score for the third language model based on the benchmark text, determining a fourth reward score for the third language model based on each of the remaining sample texts, and determining a fourth ratio between the third reward score and the sum of the plurality of fourth reward scores;

[0221] A second loss is determined according to the third ratio and the fourth ratio, and the second largest language model is trained based on the second loss.

[0222] Furthermore, the first instruction generating module 1501 is specifically configured to:

[0223] Determine the constraint weights corresponding to the constraint dimension texts, and construct a third instruction based on the initial instruction, the constraint dimension texts, and the constraint weights, wherein the third instruction is used to prompt the user to add multiple constraint condition texts to the initial instruction;

[0224] The third instruction is input into the first large language model for text generation, and the first instruction is output.

[0225] Furthermore, the first instruction generating module 1501 is further configured to:

[0226] Encode the initial instruction and the constraint dimension text respectively to obtain a first embedding vector of the initial instruction and a second embedding vector of each constraint dimension text;

[0227] Concatenate the first embedding vector with each second embedding vector to obtain multiple target embedding vectors;

[0228] The weight determination model is called to perform regression processing on each target embedding vector to obtain the constraint weight corresponding to each constraint dimension text;

[0229] The first training module 1504 is further configured to:

[0230] The second largest language model and the weight determination model are jointly trained based on the comparison text set.

[0231] Furthermore, the constraint dimension text has multiple levels, and the first instruction generating module 1501 is further used to:

[0232] Determine a first dimension text from each constraint dimension text at the first level, and determine a second dimension text from the constraint dimension text at the Nth level associated with the first dimension text, where N is an integer and N≥2;

[0233] The first language model is called to add multiple constraint texts to the initial instruction based on the second dimension text, and the first instruction is output.

[0234] Furthermore, the first instruction generating module 1501 is further configured to:

[0235] Performing semantic recognition on the initial instruction to obtain a first semantic label of the initial instruction;

[0236] Obtaining a second semantic label preset for each constraint dimension text at the first level, and matching the second semantic label with the first semantic label;

[0237] When the second semantic tag is consistent with the first semantic tag, the constraint dimension text is determined to be the first dimension text.

[0238] Furthermore, the text generation device further includes a verification module (not identified), which is specifically configured to:

[0239] Call the first language model to verify the sample text and output the verification results and explanation text;

[0240] The verification result is used to indicate the integrity of the sample text, and the explanation text is used to explain the verification result.

[0241] Furthermore, the text generation device further includes a text refinement module (not labeled). After the first language model is called to verify the first instruction and the verification result and the interpretation text are output, the text refinement module is specifically configured to:

[0242] When the verification result indicates that the sample text is incomplete, the first large language model is called to regenerate the text based on the first instruction and the second instruction respectively until a complete sample text is output.

[0243] In summary, the text generation device 1500 in the embodiment of the present application obtains an initial instruction and a constraint dimension text, calls the first language model to add multiple constraint texts to the initial instruction based on the constraint dimension text, thereby outputting a more complex first instruction, then calls the first language model to delete at least one constraint text from the first instruction, outputs a second instruction, calls the first language model to generate text based on the first instruction and the second instruction respectively, and outputs sample texts corresponding to the first instruction and the second instruction respectively. Since the complicated first instruction is more accurate and specific, the quality of the generated sample text is better. However, since the second instruction contains fewer types of constraint texts than the first instruction, the quality of the generated sample text is slightly worse. Therefore, by combining the sample texts, a comparative text set consisting of sample texts with different qualities can be obtained. Subsequently, the second language model can be trained based on the comparative text set to allow the second language model to learn a better quality text generation method, thereby improving the second language model's ability to follow instructions. Subsequently, when the target instruction is obtained and the trained second language model is called to generate text based on the target instruction, the quality of the generated target text can be effectively improved.

[0244] Reference Figure 16 , Figure 16 This is an optional structural diagram of a model training device provided in an embodiment of the present application. The model training device 1600 includes:

[0245] The second instruction generation module 1601 is configured to obtain an initial instruction and a constraint dimension text, invoke the first language model to add multiple constraint condition texts to the initial instruction based on the constraint dimension text, and output a first instruction;

[0246] A second instruction rewriting module 1602 is configured to call the first large language model to delete at least one constraint text from the first instruction and output a second instruction;

[0247] The second sample generation module 1603 is configured to call the first large language model to generate text based on the first instruction and the second instruction, and output sample texts corresponding to the first instruction and the second instruction respectively;

[0248] The second training module 1604 is configured to combine sample texts to obtain a comparison text set, and train the second largest language model based on the comparison text set.

[0249] In summary, the model training device 1600 in the embodiment of the present application obtains the initial instruction and the constraint dimension text, calls the first language model to add multiple constraint texts to the initial instruction based on the constraint dimension text, thereby outputting a more complex first instruction, and then calls the first language model to delete at least one constraint text from the first instruction, outputting the second instruction, and calls the first language model to generate text based on the first instruction and the second instruction respectively, and outputs the sample text corresponding to the first instruction and the second instruction respectively. Since the complicated first instruction is more accurate and specific, the quality of the generated sample text is better, and since the second instruction contains fewer types of constraint texts than the first instruction, the quality of the generated sample text is slightly worse. Therefore, by combining the sample texts, a comparative text set consisting of sample texts with different qualities can be obtained, and then the second language model can be trained based on the comparative text set, so that the second language model can learn a better quality text generation method, thereby improving the second language model's ability to follow the instructions.

[0250] The electronic device for executing the above-mentioned text generation method or model training method provided in the embodiment of the present application may be a terminal, referring to Figure 17 , Figure 17 This is a partial structural block diagram of a terminal provided in an embodiment of the present application. The terminal includes: a camera assembly 1710, a first memory 1720, an input unit 1730, a display unit 1740, a sensor 1750, an audio circuit 1760, a wireless fidelity (WiFi) module 1770, a first processor 1780, and a first power supply 1790. It will be understood by those skilled in the art that Figure 17 The terminal structure shown in the figure does not constitute a limitation to the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0251] The camera assembly 1710 can be used to capture images or videos. Optionally, the camera assembly 1710 includes a front camera and a rear camera. Typically, the front camera is provided on the front panel of the terminal, and the rear camera is provided on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions.

[0252] The first memory 1720 may be used to store software programs and modules. The first processor 1780 executes the software programs and modules stored in the first memory 1720 to perform various functional applications and data processing of the terminal.

[0253] The input unit 1730 may be configured to receive input digital or character information and generate key signal input related to the terminal's settings and function control. Specifically, the input unit 1730 may include a touch panel 1731 and other input devices 1732.

[0254] The display unit 1740 may be configured to display input information or provided information and various menus of the terminal. The display unit 1740 may include a display panel 1741 .

[0255] The audio circuit 1760 , the speaker 1761 , and the microphone 1762 may provide an audio interface.

[0256] The first power source 1790 may be alternating current, direct current, a disposable battery, or a rechargeable battery.

[0257] The number of sensors 1750 can be one or more, and the one or more sensors 1750 include but are not limited to: acceleration sensors, gyroscope sensors, pressure sensors, optical sensors, etc. Among them:

[0258] The accelerometer can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal. For example, the accelerometer can be used to detect the components of gravity acceleration on the three coordinate axes. The first processor 1780 can control the display unit 1740 to display the user interface in a horizontal or vertical view based on the gravity acceleration signal collected by the accelerometer. The accelerometer can also be used to collect game or user motion data.

[0259] The gyroscope sensor can detect the terminal's orientation and rotation angle. It can also work with the accelerometer to capture the user's 3D movements on the terminal. Based on the data collected by the gyroscope sensor, the first processor 1780 can implement the following functions: motion sensing (such as changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0260] The pressure sensor can be set on the side frame of the terminal and / or the lower layer of the display unit 1740. When the pressure sensor is set on the side frame of the terminal, it can detect the user's grip signal of the terminal, and the first processor 1780 performs left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor. When the pressure sensor is set on the lower layer of the display unit 1740, the first processor 1780 controls the operability controls on the UI interface based on the user's pressure operation on the display unit 1740. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0261] The optical sensor is used to collect ambient light intensity. In one embodiment, the first processor 1780 can control the display brightness of the display unit 1740 based on the ambient light intensity collected by the optical sensor. Specifically, when the ambient light intensity is high, the display brightness of the display unit 1740 is increased; when the ambient light intensity is low, the display brightness of the display unit 1740 is decreased. In another embodiment, the first processor 1780 can also dynamically adjust the shooting parameters of the camera assembly 1710 based on the ambient light intensity collected by the optical sensor.

[0262] In this embodiment, the first processor 1780 included in the terminal can execute the text generation method or model training method of the previous embodiment.

[0263] The electronic device for executing the above-mentioned text generation method or model training method provided in the embodiment of the present application may also be a server, referring to Figure 18 , Figure 18 This is a partial structural block diagram of a server provided in an embodiment of the present application. The server 1800 may have relatively large differences due to different configurations or performances, and may include one or more second processors 1822 and a second memory 1832, and one or more storage media 1830 (e.g., one or more mass storage devices) storing application programs 1842 or data 1844. Among them, the second memory 1832 and the storage medium 1830 may be temporary storage or permanent storage. The program stored in the storage medium 1830 may include one or more modules (not shown in the figure), each module may include a series of instruction operations in the server 1800. Furthermore, the second processor 1822 may be configured to communicate with the storage medium 1830 to execute a series of instruction operations in the storage medium 1830 on the server 1800.

[0264] The server 1800 may also include one or more second power supplies 1826, one or more wired or wireless network interfaces 1850, one or more input and output interfaces 1858, and / or one or more operating systems 1841, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0265] The second processor 1822 in the server 1800 can be used to execute the text generation method or the model training method.

[0266] An embodiment of the present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute the text generation method or model training method of each of the aforementioned embodiments.

[0267] The present application also provides a computer program product, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to implement the above-described text generation method or model training method.

[0268] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the numbers used in this way can be interchanged where appropriate to describe the embodiments of the present application, for example, they can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0269] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0270] It should be understood that in the description of the embodiments of the present application, multiple (or multiple items) means more than two, greater than, less than, exceed, etc. are understood to exclude the number itself, and above, below, within, etc. are understood to include the number itself.

[0271] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0272] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0273] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0274] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0275] It should also be understood that the various implementation methods provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0276] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the art can also make various equivalent modifications or substitutions under the shared conditions that do not violate the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A text generation method, characterized in that: include: Obtaining an initial instruction and a constraint dimension text, calling a first language model to add a plurality of constraint condition texts to the initial instruction based on the constraint dimension text, and outputting a first instruction; calling the first language model to delete at least one constraint text from the first instruction, and outputting a second instruction; Calling the first language model to generate text based on the first instruction and the second instruction, and outputting sample texts corresponding to the first instruction and the second instruction respectively; Combining the sample texts to obtain a comparison text set, and training a second language model based on the comparison text set; Obtain a target instruction, call the trained second language model to generate text based on the target instruction, and output the target text.

2. The text generation method according to claim 1, characterized in that The first language model outputs multiple types of the second instructions, where the different types of the second instructions are obtained by deleting different amounts of the constraint texts, and the combining of the sample texts to obtain a comparison text set includes: Taking each of the sample texts as a reference text, combining the reference text with at least one remaining sample text to obtain a comparison document set; The number of the constraint condition texts satisfied by the reference text is greater than the number of the constraint condition texts satisfied by the remaining sample texts.

3. The text generation method according to claim 2, characterized in that The comparison document set includes the benchmark text and one of the remaining sample texts, and the training of the second language model based on the comparison document set includes: Determining a first reward score for a second language model based on the benchmark text, determining a second reward score for the second language model based on the remaining sample text, and determining a first ratio between the first reward score and the second reward score; Determining a third reward score of a third language model based on the benchmark text, determining a fourth reward score of the third language model based on the remaining sample text, and determining a second ratio between the third reward score and the fourth reward score; A first loss is determined according to the first ratio and the second ratio, and the second large language model is trained based on the first loss.

4. The text generation method according to claim 2, characterized in that The comparison document set includes the benchmark text and the remaining plurality of sample texts, and the training of the second language model based on the comparison document set includes: Determining a first reward score for a second language model based on the benchmark text, determining a second reward score for the second language model based on each of the remaining sample texts, and determining a third ratio between the first reward score and a sum of the plurality of second reward scores; Determining a third reward score of a third language model based on the benchmark text, determining a fourth reward score of the third language model based on each of the remaining sample texts, and determining a fourth ratio between the third reward score and a sum of the plurality of fourth reward scores; A second loss is determined according to the third ratio and the fourth ratio, and the second large language model is trained based on the second loss.

5. The text generation method according to claim 1, characterized in that The calling of the first language model adds a plurality of constraint condition texts to the initial instruction based on the constraint dimension text, and outputs a first instruction, including: Determining the constraint weights corresponding to the constraint dimension texts, and constructing a third instruction based on the initial instruction, the constraint dimension texts, and the constraint weights, wherein the third instruction is used to prompt the user to add multiple constraint condition texts to the initial instruction; The third instruction is input into the first large language model for text generation, and the first instruction is output.

6. The text generation method according to claim 5, characterized in that Determining the constraint weight corresponding to each constraint dimension text includes: Encoding the initial instruction and the constraint dimension text respectively to obtain a first embedding vector of the initial instruction and a second embedding vector of each constraint dimension text; Concatenate the first embedding vector with each of the second embedding vectors to obtain multiple target embedding vectors; Calling the weight determination model to perform regression processing on each of the target embedding vectors to obtain the constraint weight corresponding to each of the constraint dimension texts; The training of the second language model based on the comparison text set includes: The second largest language model and the weight determination model are jointly trained based on the comparison text set.

7. The text generation method according to claim 1, characterized in that The constraint dimension text has multiple levels. The calling of the first language model adds multiple constraint condition texts to the initial instruction based on the constraint dimension text to output a first instruction, including: Determine a first dimensional text from each of the constraint dimensional texts at the first level, and determine a second dimensional text from the constraint dimensional text at the Nth level associated with the first dimensional text, where N is an integer, N≥2; The first language model is called to add a plurality of constraint condition texts to the initial instruction based on the second dimensional text, and a first instruction is output.

8. The text generation method according to claim 7, characterized in that: The determining of the first dimension text from the constraint dimension texts at the first level includes: Performing semantic recognition on the initial instruction to obtain a first semantic label of the initial instruction; Obtaining a second semantic tag preset for each of the constraint dimension texts at the first level, and matching the second semantic tag with the first semantic tag; When the second semantic tag is consistent with the first semantic tag, the constraint dimensional text is determined to be the first dimensional text.

9. The text generation method according to any one of claims 1 to 8, characterized in that: After calling the first language model to generate text based on the first instruction and the second instruction respectively, and outputting sample texts corresponding to the first instruction and the second instruction respectively, the text generation method further includes: Calling the first language model to verify the sample text, and outputting a verification result and an explanation text; The verification result is used to indicate the integrity of the sample text, and the explanation text is used to explain the verification result.

10. The text generation method according to claim 9, characterized in that: After calling the first language model to verify the first instruction and outputting the verification result and the interpretation text, the text generation method further includes: When the verification result indicates that the sample text is incomplete, the first large language model is called to regenerate the text based on the first instruction and the second instruction respectively until the complete sample text is output.

11. A model training method, characterized in that: include: Obtaining an initial instruction and a constraint dimension text, calling a first language model to add a plurality of constraint condition texts to the initial instruction based on the constraint dimension text, and outputting a first instruction; calling the first language model to delete at least one constraint text from the first instruction, and outputting a second instruction; Calling the first language model to generate text based on the first instruction and the second instruction, and outputting sample texts corresponding to the first instruction and the second instruction respectively; The sample texts are combined to obtain a comparison text set, and a second language model is trained based on the comparison text set.

12. A text generation device, characterized in that: include: A first instruction generation module is configured to obtain an initial instruction and a constraint dimension text, invoke a first language model to add a plurality of constraint condition texts to the initial instruction based on the constraint dimension text, and output a first instruction; a first instruction rewriting module, configured to call the first large language model to delete at least one constraint text from the first instruction and output a second instruction; A first sample generation module is configured to call the first language model to generate text based on the first instruction and the second instruction, and output sample texts corresponding to the first instruction and the second instruction respectively; A first training module is configured to combine the sample texts to obtain a comparison text set, and train a second language model based on the comparison text set; The text generation module is used to obtain the target instruction, call the trained second language model to generate text based on the target instruction, and output the target text.

13. A model training device, characterized in that: include: A second instruction generation module is configured to obtain an initial instruction and a constraint dimension text, invoke the first language model to add a plurality of constraint condition texts to the initial instruction based on the constraint dimension text, and output a first instruction; a second instruction rewriting module, configured to call the first large language model to delete at least one constraint text from the first instruction and output a second instruction; A second sample generation module is configured to call the first language model to generate text based on the first instruction and the second instruction, and output sample texts corresponding to the first instruction and the second instruction respectively; The second training module is used to combine the sample texts to obtain a comparison text set, and train the second largest language model based on the comparison text set.

14. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, it implements the text generation method described in any one of claims 1 to 10, or implements the model training method described in claim 11.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the text generation method described in any one of claims 1 to 10, or implements the model training method described in claim 11.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the text generation method described in any one of claims 1 to 10, or implements the model training method described in claim 11.

Images