Large model training method and device, text generation method and device, equipment and medium

Abstract

The invention provides a large model training method and device, a text generation method and device, equipment and a medium, relates to the technical field of artificial intelligence, in particular to the technical fields of natural language processing, deep learning and the like, and can be used for application scenes such as generative retrieval, document intelligent editing, intelligent assistants, virtual assistants, intelligent e-commerce and the like. The training method comprises the steps that a plurality of sample dialogue texts and a plurality of corresponding annotation question recommendation texts are obtained, and the sample dialogue texts belong to a plurality of dialogue categories; multiple first prompt texts corresponding to the multiple dialogue categories are determined, and the first prompt texts are used for indicating the first large model to generate prediction problem recommendation texts conforming to the dialogue categories corresponding to the first prompt texts; and adjusting parameters of the first large model by utilizing the plurality of first prompt texts, the plurality of sample dialogue texts and the plurality of annotation question recommendation texts.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, especially to technical fields such as natural language processing and deep learning, and can be used in application scenarios such as generative retrieval, intelligent document editing, intelligent assistants, virtual assistants, and intelligent e-commerce. It specifically relates to a large model training method, a text generation method, a large model training device, a text generation device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include natural language processing technology, computer vision technology, speech recognition technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0003] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0004] The present disclosure provides a large model training method, a text generation method, a large model training device, a text generation device, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of the present disclosure, a large model training method is provided, comprising: obtaining a plurality of sample dialogue texts and a corresponding plurality of annotated question recommendation texts, wherein the plurality of sample dialogue texts belong to a plurality of dialogue categories; determining a plurality of first prompt texts corresponding to the plurality of dialogue categories, wherein the first prompt text is used to instruct a first large model to generate a predicted question recommendation text that conforms to the dialogue category corresponding to the first prompt text; and adjusting the parameters of the first large model using the plurality of first prompt texts, the plurality of sample dialogue texts, and the plurality of annotated question recommendation texts.

[0006] According to another aspect of the present disclosure, a text generation method is provided, comprising: obtaining a target dialogue text and determining a target dialogue category of the target dialogue text; and inputting the target dialogue text and a first prompt text corresponding to the target dialogue category into a large model to obtain a question recommendation text, wherein the large model is trained using the above-mentioned large model training method.

[0007] According to another aspect of the present disclosure, a training device for a large model is provided, comprising: a first acquisition unit, configured to acquire a plurality of sample dialogue texts and a corresponding plurality of annotated question recommendation texts, wherein the plurality of sample dialogue texts belong to a plurality of dialogue categories; a determination unit, configured to determine a plurality of first prompt texts corresponding to the plurality of dialogue categories, wherein the first prompt texts are used to instruct a first large model to generate a predicted question recommendation text that conforms to the dialogue category corresponding to the first prompt text; and a first parameter adjustment unit, configured to adjust parameters of the first large model using the plurality of first prompt texts, the plurality of sample dialogue texts, and the plurality of annotated question recommendation texts.

[0008] According to another aspect of the present disclosure, there is provided a text generation device, comprising: a fourth acquisition unit, configured to acquire a target dialogue text and determine a target dialogue category of the target dialogue text; and a fourth large model input unit, configured to input the target dialogue text and a first prompt text corresponding to the target dialogue category into a large model in step to obtain a question recommendation text, wherein the large model is trained using the above-mentioned large model training device.

[0009] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above method.

[0010] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above method.

[0011] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0012] According to one or more embodiments of the present disclosure, the present disclosure obtains sample conversation texts belonging to multiple conversation categories and corresponding annotated question recommendation texts, and then determines the first prompt texts corresponding to the multiple conversation categories, thereby connecting the conversation texts of multiple categories to the same large generation model to generate question recommendation texts, thereby reducing business costs and improving the coverage of question recommendations for conversations of different categories.

[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0015] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0016] Figure 2 A flowchart of a method for training a large model according to an embodiment of the present disclosure is shown;

[0017] Figure 3 A flowchart of a method for training a large model according to an embodiment of the present disclosure is shown;

[0018] Figure 4 A flowchart of obtaining multiple sample conversation texts and corresponding multiple annotated question recommendation texts according to an embodiment of the present disclosure is shown;

[0019] Figure 5 A flowchart of a method for training a large model according to an embodiment of the present disclosure is shown;

[0020] Figure 6 A flowchart of a text generation method according to an embodiment of the present disclosure is shown;

[0021] Figure 7 A structural block diagram of a large model training device according to an embodiment of the present disclosure is shown;

[0022] Figure 8 A structural block diagram of a text generation device according to an embodiment of the present disclosure is shown; and

[0023] Fig. 9 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0024] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0025] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0026] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0027] In the related art, the business cost of existing question recommendation services is relatively high.

[0028] To solve the above problems, the present invention obtains sample conversation texts belonging to multiple conversation categories and corresponding annotated question recommendation texts, and then determines the first prompt texts corresponding to the multiple conversation categories, thereby connecting the conversation texts of multiple categories to the same large generation model to generate question recommendation texts, thereby reducing business costs and improving the coverage of question recommendations for conversations of different categories.

[0029] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0030] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0031] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable execution of the methods of the present disclosure.

[0032] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0033] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0034] The user can use client devices 101, 102, 103, 104, 105 and / or 106 to perform human-computer interaction. The client device can provide an interface that enables the user of the client device to interact with the client device. The client device can also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.

[0035] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing various different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0036] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0037] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0038] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0039] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and / or 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and / or 106.

[0040] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0041] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The databases 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The databases 130 may be of different types. In some embodiments, the databases used by the server 120 may be, for example, relational databases. One or more of these databases may store, update, and retrieve data to and from the databases in response to commands.

[0042] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0043] Figure 1The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0044] According to one aspect of the present disclosure, a large model training method is provided. Figure 2 As shown, the training method includes: step S201, obtaining multiple sample dialogue texts and corresponding multiple annotated question recommendation texts, wherein the multiple sample dialogue texts belong to multiple dialogue categories; step S202, determining multiple first prompt texts corresponding to the multiple dialogue categories, wherein the first prompt text is used to instruct the first large model to generate a predicted question recommendation text that conforms to the dialogue category corresponding to the first prompt text; and step S203, adjusting the parameters of the first large model using the multiple first prompt texts, the multiple sample dialogue texts and the multiple annotated question recommendation texts.

[0045] Therefore, by obtaining sample conversation texts belonging to multiple conversation categories and the corresponding annotated question recommendation texts, and then determining the first prompt texts corresponding to the multiple conversation categories, it is possible to connect multiple categories of conversation texts to the same large generation model to generate question recommendation texts, thereby reducing business costs and improving the coverage of question recommendations for different categories of conversations.

[0046] In the present disclosure, the big model is a deep learning big model, which has end-to-end characteristics and can directly generate reply data based on the user's input data without the help of functional components or other inputs outside the deep learning big model. In other words, the deep learning big model itself has a generation function. The deep learning big model can be a big language model. A big language model generally refers to a deep learning big model with billions or even hundreds of billions of parameters, which are usually trained on large-scale text data or data of other modalities. The big language model can be used for various natural language processing tasks, such as text generation, language translation, and question-answering systems.

[0047] The deep learning large model can, for example, adopt an N-layer Transformer network structure with an encoder (Encoder) and a decoder (Decoder), or a unified pre-trained language model (Unified pre-trained LanguageModel, UniLM) network structure. It is understandable that the deep learning large model can also be other neural network models based on the Transformer network structure, which is not limited here. The input and output of the deep learning large model are both composed of identifiers (tokens). Each identifier can correspond to a single word, character, word or special symbol, as described below. The deep learning large model can be trained using pre-training tasks and generation tasks to have the above-mentioned generation function.

[0048] The training method proposed in this disclosure is used to train the large model's ability to generate question recommendation text (also referred to as question recommendation or Sug in this disclosure). In question recommendation, how can the question recommendation system predict and generate the user's next possible query text (Query) based on the user dialogue, ensure the generation effect while improving the question recommendation coverage, thereby increasing the impact of the question recommendation strategy, improving user experience and driving the growth of product core indicators.

[0049] In other words, the corresponding first prompt text is written for each dialogue category, the training data is organized, and the Mixture-of-Experts (MoE) is simulated through different first prompt texts to solve the problem of low coverage caused by the single category and high cost of generating question recommendations (Sug) using large models.

[0050] Question recommendation refers to the recommendation system predicting the questions that the user may ask in the next round of dialogue based on the user's questions and the dialogue system's answers in the previous round of dialogue, and finally presenting them on the client for the user to select and click. The dialogue system can be implemented by a large model. The large model used for the dialogue system can be different from the first large model used to generate question recommendation text or the second large model used to generate labeled data, as will be introduced below. The recommended words or other forms of recommended content predicted by the recommendation system are called question recommendations.

[0051] In step S201, a plurality of sample dialogue texts and a plurality of corresponding annotated question recommendation texts are obtained. The plurality of sample dialogue texts belong to a plurality of dialogue categories.

[0052] According to some embodiments, the multiple dialogue categories may include knowledge questions and answers, text creation, and small talk. In addition to the above categories, the dialogue categories may also include other content, which is not limited here.

[0053] For each dialogue category, one or more sample dialogue texts may be obtained. The sample dialogue text may include a user query text (Query) or a question sentence, and a dialogue system answer text or a reply sentence. The annotated question recommendation text corresponding to the sample dialogue text may be used as the ground truth corresponding to the question recommendation task and the sample dialogue text. In other words, the first large model may be used to adjust parameters of the first large model (for example, by fine-tuning or other training methods) using the predicted question recommendation text and the multiple annotated question recommendation texts generated by the first large model for multiple sample dialogue texts, as will be described below.

[0054] In step S202, multiple first prompt texts corresponding to multiple dialogue categories are determined. The prompt text uses natural language to instruct the large model to complete a specific task and needs to meet specific detail requirements. The first prompt text needs to instruct the first large model to generate a question recommendation text that conforms to the corresponding dialogue category. By constructing first prompt texts corresponding to multiple dialogue categories, the understanding and generation capabilities of the first large model can be fully utilized, so that a single large model can be used to generate question recommendation texts that conform to the corresponding dialogue categories for different dialogue categories.

[0055] In some embodiments, the first prompt text may include a task description text. The task description text includes at least a conversation category corresponding to the first prompt text. The task description text may also include an identity corresponding to the conversation category.

[0056] In some embodiments, the first prompt text may include question restriction text, which describes content requirements and format requirements.

[0057] In some embodiments, the first prompt text may include a questioning idea text. The questioning idea text describes at least one questioning direction. Exemplary questioning directions may include making recommendations in combination with scenarios.

[0058] In some embodiments, the first prompt text may include at least one case, wherein each case includes an example user query text, an example dialogue system answer text, and at least one example question recommendation text.

[0059] In some embodiments, the first prompt text may further include format requirement text, which describes the format of the result output by the large model.

[0060] In some embodiments, the first large model can use the small model mT5-Large. mT5-Large is a multilingual version of T5, with powerful sequence-to-sequence (Seq2Seq) generation capabilities, and excellent performance on multilingual tasks. With only 1.2B parameters, the generation speed is fast enough to meet the time requirements of the question recommendation business while ensuring high-quality generation effects. The mT5 structure is based on Transformers, and its encoder is an encoder (Encoder) similar to BERT based on multi-head self-attention, which is responsible for embedding and understanding the input information and passing the information to the decoder; the decoder is a decoder (Decoder) structure based on masked self-attention, which is responsible for using the input information to generate the corresponding output text; the two combine features through a cross-attention mechanism.

[0061] It is understandable that the first large model may also adopt other model structures, which are not limited here.

[0062] According to some embodiments, step S203, adjusting the parameters of the first large model using multiple first prompt texts, multiple sample dialogue texts and multiple annotated question recommendation texts may include: for a second sample dialogue text among the multiple sample dialogue texts, inputting the second sample dialogue text and the first prompt text corresponding to the second dialogue category into the first large model to obtain a predicted question recommendation text corresponding to the second sample dialogue text, wherein the second sample dialogue text belongs to the second dialogue category; and adjusting the parameters of the first large model based on the predicted question recommendation text and the annotated text recommendation text corresponding to the second sample dialogue text.

[0063] In the above training process, the training strategy, training parameters, and testing methods can be determined according to the needs, and are not limited here. It is understandable that in addition to the above method, other methods can be used to adjust the parameters of the first large model using multiple first prompt texts, multiple sample dialogue texts, and multiple annotated question recommendation texts.

[0064] According to some embodiments, Figure 3 As shown, the training method may also include: step S304, obtaining multiple second test dialogue texts; step S305, inputting the multiple second test dialogue texts and the corresponding first prompt texts into the adjusted first large model to obtain multiple second test question recommendation texts; and step S306, in response to determining that the multiple second test question recommendation texts do not meet the third preset requirement, re-adjusting the parameters of the first large model. It can be understood that Figure 3 Steps S301 to S304 in the above description may refer to the above description. Figure 2 The description of steps S201 to S204 in the process will not be repeated here.

[0065] After adjusting the parameters of the first large model, the first large model can be tested using the second test dialogue text. If the question recommendation text output by the first large model does not meet the requirements, the parameters of the first large model can be readjusted. Thus, through the above method, the test evaluation and update adjustment of the first large model are achieved.

[0066] In some embodiments, the third preset requirement may include offline test indicators and / or online test indicators. Offline test indicators may include relevance, diversity, coverage, fluency, and other evaluation indicators for natural language texts, and may also include manual evaluation indicators. Online test indicators may include click-through rate, conversion rate, user retention rate, etc.

[0067] According to some embodiments, the plurality of second test dialogue texts may include a target test dialogue text, and the target test dialogue text may correspond to a test dialogue category in the plurality of dialogue categories. Step S306, in response to determining that the plurality of second test question recommendation texts do not meet the third preset requirement, readjusting the parameters of the first large model may include: in response to determining that the target test question recommendation text in the plurality of second test question recommendation texts does not meet the third preset requirement, optimizing the first prompt text corresponding to the test dialogue category, the target test question recommendation text being obtained by inputting the target test dialogue text and the first prompt text corresponding to the test dialogue category into the adjusted first large model; and using the optimized first prompt text to readjust the parameters of the first large model.

[0068] When it is determined that the question recommendation text output by the first large model does not meet the third preset requirement, it can be determined which dialogue category has the poor output result, and the first prompt text corresponding to the dialogue category can be adjusted. The adjusted first prompt text can be used to readjust the parameters of the first large model to improve the first large model's ability to generate question recommendation text for the dialogue category.

[0069] In some embodiments, optimizing the first prompt text may include determining a first target question direction based on the target test question recommendation text; and adding the first target question direction to the question idea text in the first prompt text.

[0070] According to some embodiments, step S306, in response to determining that multiple second test question recommended texts do not meet the third preset requirement, readjusting the parameters of the first large model may include: adjusting the ratio of sample dialogue texts corresponding to each of the multiple dialogue categories in the multiple sample dialogue texts; and using the multiple sample dialogue texts after the adjusted ratio, readjusting the parameters of the first large model.

[0071] When it is determined that the question recommendation text output by the first large model does not meet the third preset requirement, the distribution of training data is optimized by dynamically adjusting the sample ratio of different dialogue categories, thereby enhancing the generalization ability of the large model and improving the coverage of multi-category requirements.

[0072] In some embodiments, for data matching, it is necessary to consider the online performance of the model and the user situation. First, the conversation data of users within a certain period of time can be collected separately, and the number of times each category of conversation (QA) appears can be counted. The training data is constructed according to the proportion of this number. If the model does not meet the expected indicators after it goes online, the click-through rate of each category during the online experiment can be counted. For conversation categories with low click-through rates, more data of this conversation category is added to the training data, and the training data is cleaned up in a targeted manner according to the problems that occurred in the question recommendation (Sug) in the experiment.

[0073] In step S306, generation parameters or training parameters may also be adjusted.

[0074] In some embodiments, a batch of dialogue texts can be randomly selected for generation parameters, and the large model can be used to generate them in an offline environment. The default generation parameters of the large model are used as a baseline, and the generation parameters are adjusted for control experiments to obtain multiple sets of data (e.g., dozens of sets), and the question recommendation texts are evaluated (e.g., according to quality and diversity), and the set with the best evaluation effect is selected as the generation parameters for final use.

[0075] In some embodiments, in terms of training parameters, three sets of parameters, namely, the more conservative, the most commonly used, and the more radical, can be selected to train the large model, and the large model can be evaluated offline and / or online, and finally the set with the best effect can be selected as the training parameters for subsequent large model training.

[0076] Returning to step S201, the plurality of labeled question recommendation texts may be generated using another large model.

[0077] According to some embodiments, Figure 4 As shown, step S201, obtaining multiple sample dialogue texts and corresponding multiple labeled question recommendation texts may include: step S401, determining multiple second prompt texts corresponding to multiple dialogue categories, wherein the second prompt text is used to instruct the trained second largest model to generate a labeled question recommendation text that conforms to the dialogue category corresponding to the second prompt text; and step S402, for a first sample dialogue text among the multiple sample dialogue texts, inputting the first sample dialogue text and the second prompt text corresponding to the first dialogue category among the multiple dialogue categories into the second largest model to obtain a first labeled question recommendation text corresponding to the first sample dialogue text, wherein the first sample dialogue text belongs to the first dialogue category.

[0078] Through the above method, the trained second largest model can be effectively used to generate labeled data, thereby realizing the distillation of the first largest model to the second largest model.

[0079] According to some embodiments, the first largest model has a smaller number of parameters than the second largest model. The second largest model may be a large language model with stronger generation, reasoning, and creation capabilities, which may have a higher number of parameters, and accordingly, its training and deployment costs are much greater than the first largest model. By using the second largest model with a larger number of parameters for annotation, and using these high-quality annotated question recommendation texts to train the first largest model, knowledge distillation is achieved, so that the first largest model can provide a recommendation effect close to that of the second largest model at a lower cost for the task of generating recommendations for questions in multiple dialogue categories.

[0080] In some embodiments, the second largest model may use the ERNIE Bot 4.0 (EB4) model. EB4 is based on the Transformer architecture and has tens of billions of parameters. It integrates multimodal understanding and generation capabilities, supports multiple input forms such as text and images, and can provide high accuracy and fluency in natural language processing tasks. EB4 is widely used in intelligent customer service, content creation and other fields, significantly improving user interaction experience and system efficiency.

[0081] It is understandable that the second largest model may also adopt other large language models, which is not limited here.

[0082] In some embodiments, by constructing second prompt texts corresponding to multiple dialogue categories, the understanding and generation capabilities of the second large model can be fully utilized, so that a single large model can be used to generate question recommendation texts that meet the corresponding dialogue categories for different dialogue categories.

[0083] In some embodiments, the second prompt text may include a task description text. The task description text includes at least a conversation category corresponding to the second prompt text. The task description text may also include an identity corresponding to the conversation category.

[0084] In some embodiments, the second prompt text may include question restriction text, which describes content requirements and format requirements.

[0085] In some embodiments, the second prompt text may include a questioning idea text. The questioning idea text describes at least one questioning direction. Exemplary questioning directions may include making recommendations in combination with scenarios.

[0086] In some embodiments, the second prompt text may include at least one case, each of which includes an example user query text, an example dialogue system answer text, and at least one example question recommendation text.

[0087] In some embodiments, the second prompt text may further include format requirement text, which describes the format of the result output by the large model.

[0088] According to some embodiments, the first dialogue category may include multiple dialogue subcategories. The second prompt text corresponding to the first dialogue category may include multiple subcategory prompt texts corresponding to the multiple dialogue subcategories, and the subcategory prompt texts are used to instruct the second large model to generate a labeling question recommendation text that conforms to the corresponding dialogue subcategory. The first sample dialogue text belongs to the first dialogue subcategory among the multiple dialogue subcategories, and the first sample dialogue text and the subcategory prompt text corresponding to the first dialogue subcategory are input into the second large model to obtain the first labeling question recommendation text.

[0089] By refining the first dialogue category into multiple dialogue subcategories, the number and coverage of training samples of the first large model can be enriched, thereby significantly improving the subsequent training effect of the first large model. It should be noted that the second large model can know the dialogue subcategory corresponding to the sample dialogue text when annotating, but the first prompt text does not contain information related to the dialogue subcategory. In other words, under the same dialogue category, sample dialogue texts of different dialogue subcategories use the same first prompt text when using the first large model to generate predicted question recommendation texts, that is, the first prompt text corresponding to the dialogue category to which these different dialogue subcategories belong. This approach can force the first large model to learn the content corresponding to different dialogue subcategories, while improving the generalization of the first large model.

[0090] In an exemplary embodiment, the first conversation category may be small talk, which may include two conversation subcategories: content recommendation and daily discussion. It is understandable that the conversation category "small talk" may also include other conversation subcategories, and other conversation categories may include corresponding multiple conversation subcategories, which are not limited here.

[0091] In some embodiments, the subcategory prompt text may include the above-mentioned task description text, question restriction text, question idea text and / or at least one case. The task description text in the subcategory prompt text may include the corresponding dialogue category and / or the corresponding dialogue subcategory.

[0092] According to some embodiments, Figure 5 As shown, the training method may also include: step S504, obtaining a first test dialogue text corresponding to the first dialogue subcategory; step S505, inputting the first test dialogue text and the first prompt text corresponding to the first dialogue category into the adjusted first large model to obtain a first test question recommendation text; step S506, in response to determining that the first test question recommendation text does not meet the first preset requirement, optimizing the subcategory prompt text corresponding to the first dialogue subcategory; step S507, inputting the optimized subcategory prompt text and the first sample dialogue text into the second large model to obtain a second annotated question recommendation text; and step S508, using the first sample dialogue text, the first prompt text corresponding to the first dialogue category, and the second annotated question recommendation text to readjust the parameters of the first large model.

[0093] Understandably, Figure 5 The operations and effects of steps S501 to S503 in the above description can be referred to. Figure 2 The description of steps S201 to S203 in the process will not be repeated here.

[0094] Through the above method, the two parts of using the second largest model for data annotation and training the first largest model can be coupled, and the generation capability of the first largest model for recommending questions of different subcategories can be effectively improved.

[0095] In some embodiments, in step S506, optimizing the subcategory prompt text corresponding to the first dialogue subcategory may include determining a second target question direction based on the first test question recommendation text; and adding the second target question direction to the question idea text in the subcategory prompt text.

[0096] According to some embodiments, Figure 4 As shown, step S201, obtaining multiple sample dialogue texts and corresponding multiple labeled question recommendation texts may also include: step S403, in response to determining that the first labeled question recommendation text does not meet the second preset requirement, optimizing the second prompt text corresponding to the first dialogue category.

[0097] Therefore, through the above method, the second prompt text can be iterated to improve the quality of the annotation data.

[0098] In some embodiments, optimizing the second prompt text may include, for example, determining a third target question direction based on the second annotated question recommendation text; and adding the third target question direction to the question idea text in the second prompt text.

[0099] According to some embodiments, Figure 4 As shown, step S201, obtaining multiple sample dialogue texts and corresponding multiple annotation question recommendation texts may also include: step S404, determining a first batch of sample dialogue texts from multiple sample dialogue texts, the first batch of sample dialogue texts including a first sample dialogue text; step S405, in response to determining that the annotation question recommendation texts obtained after inputting the first batch of sample dialogue texts and the corresponding second prompt texts into the second largest model all meet the second preset requirement, determining a second batch of sample dialogue texts from the multiple sample dialogue texts, the second batch being larger than the first batch; and step S406, inputting the second batch of sample dialogue texts and the corresponding second prompt texts into the second largest model to obtain multiple annotation question recommendation texts.

[0100] In this way, the generated data can be evaluated. If it does not meet expectations, the second prompt text is tuned according to the situation in the question recommendation text, and then a small batch generation attempt is made again. If the data quality meets expectations, a large batch of annotated data is used for subsequent first large model training.

[0101] According to another aspect of the present disclosure, a text generation method is provided. Figure 6As shown, the text generation method includes: step S601, obtaining a target dialogue text and determining a target dialogue category of the target dialogue text; step S602, inputting the target dialogue text and a first prompt text corresponding to the target dialogue category into a large model to obtain a question recommendation text. The large model is trained using the large model training method provided above.

[0102] In some embodiments, the target dialogue text may include a user target query text and a dialogue system target answer text. The target dialogue category may be determined from a plurality of dialogue categories involved in the training method of the large model provided above. The large model may be the first large model that has been trained.

[0103] According to another aspect of the present disclosure, a large model training device is provided. Figure 7 As shown, the training device 700 includes: a first acquisition unit 710, configured to acquire multiple sample dialogue texts and corresponding multiple labeled question recommendation texts, wherein the multiple sample dialogue texts belong to multiple dialogue categories; a determination unit 720, configured to determine multiple first prompt texts corresponding to the multiple dialogue categories, wherein the first prompt text is used to instruct the first large model to generate a predicted question recommendation text that conforms to the dialogue category corresponding to the first prompt text; and a first parameter adjustment unit 730, configured to adjust the parameters of the first large model using the multiple first prompt texts, the multiple sample dialogue texts and the multiple labeled question recommendation texts.

[0104] It can be understood that the operations and effects of the units 710 to 730 in the device 700 can refer to the above description of steps S201 to S203, and will not be repeated here.

[0105] According to some embodiments, the multiple conversation categories may include quizzes, text creation, and small talk.

[0106] According to some embodiments, the first parameter adjustment unit may include: a third large model input subunit, configured to input the second sample dialogue text and the first prompt text corresponding to the second dialogue category into the first large model for a second sample dialogue text among multiple sample dialogue texts, so as to obtain a predicted question recommendation text corresponding to the second sample dialogue text, wherein the second sample dialogue text belongs to the second dialogue category; and a third parameter adjustment subunit, configured to adjust the parameters of the first large model based on the predicted question recommendation text and the annotated text recommendation text corresponding to the second sample dialogue text.

[0107] According to some embodiments, the training device may also include: a third acquisition unit, configured to acquire multiple second test dialogue texts; a third large model input unit, configured to input the multiple second test dialogue texts and the corresponding first prompt texts into the adjusted first large model to obtain multiple second test question recommended texts; and a third parameter adjustment unit, configured to readjust the parameters of the first large model in response to determining that the multiple second test question recommended texts do not meet the third preset requirements.

[0108] According to some embodiments, the plurality of second test dialogue texts may include a target test dialogue text, and the target test dialogue text may correspond to a test dialogue category in a plurality of dialogue categories. The third parameter adjustment unit may include: a second optimization subunit, configured to optimize a first prompt text corresponding to the test dialogue category in response to determining that a target test question recommendation text in the plurality of second test question recommendation texts does not meet a third preset requirement, the target test question recommendation text being obtained by inputting the target test dialogue text and the first prompt text corresponding to the test dialogue category into an adjusted first large model; and a first parameter adjustment subunit, configured to readjust the parameters of the first large model using the optimized first prompt text.

[0109] According to some embodiments, the third parameter adjustment unit may include: a ratio adjustment subunit, configured to adjust the ratio of sample dialogue texts corresponding to each of the multiple dialogue categories among the multiple sample dialogue texts; and a second parameter adjustment subunit, configured to readjust the parameters of the first large model using the multiple sample dialogue texts after the ratio adjustment.

[0110] According to some embodiments, the first acquisition unit may include: a first determination subunit, configured to determine multiple second prompt texts corresponding to multiple dialogue categories, respectively, wherein the second prompt text is used to instruct the trained second large model to generate a labeling question recommendation text that conforms to the dialogue category corresponding to the second prompt text; and a first large model input subunit, configured to input the first sample dialogue text and the second prompt text corresponding to the first dialogue category among the multiple dialogue categories into the second large model for a first sample dialogue text among the multiple sample dialogue texts, so as to obtain a first labeling question recommendation text corresponding to the first sample dialogue text, wherein the first sample dialogue text belongs to the first dialogue category.

[0111] According to some embodiments, the first large model may have a smaller number of parameters than the second large model.

[0112] According to some embodiments, the first dialogue category may include multiple dialogue subcategories, and the second prompt text corresponding to the first dialogue category may include multiple subcategory prompt texts corresponding to the multiple dialogue subcategories, and the subcategory prompt texts may be used to instruct the second large model to generate a labeling question recommendation text that conforms to the corresponding dialogue subcategory. The first sample dialogue text belongs to the first dialogue subcategory among the multiple dialogue subcategories, and the first sample dialogue text and the subcategory prompt text corresponding to the first dialogue subcategory may be input into the second large model to obtain the first labeling question recommendation text.

[0113] According to some embodiments, the training device may also include: a second acquisition unit, configured to acquire a first test dialogue text corresponding to a first dialogue subcategory; a first large model input unit, configured to input the first test dialogue text and the first prompt text corresponding to the first dialogue category into the adjusted first large model to obtain a first test question recommendation text; an optimization unit, configured to optimize the subcategory prompt text corresponding to the first dialogue subcategory in response to determining that the first test question recommendation text does not meet the first preset requirement; a second large model input unit, configured to input the optimized subcategory prompt text and the first sample dialogue text into the second large model to obtain a second labeled question recommendation text; and a second parameter adjustment unit, configured to readjust the parameters of the first large model using the first sample dialogue text, the first prompt text corresponding to the first dialogue category, and the second labeled question recommendation text.

[0114] According to some embodiments, the first acquisition unit may include: a first optimization subunit configured to optimize a second prompt text corresponding to the first dialogue category in response to determining that the first labeled question recommended text does not meet a second preset requirement.

[0115] According to some embodiments, the first acquisition unit may include: a second determination subunit, configured to determine a first batch of sample dialogue texts from a plurality of sample dialogue texts, the first batch of sample dialogue texts including a first sample dialogue text; a third determination subunit, configured to determine a second batch of sample dialogue texts from a plurality of sample dialogue texts in response to determining that the annotation question recommendation texts obtained after inputting the first batch of sample dialogue texts and the corresponding second prompt texts into a second large model respectively meet a second preset requirement, the second batch being larger than the first batch; and a second large model input subunit, configured to input the second batch of sample dialogue texts and the corresponding second prompt texts into the second large model respectively to obtain a plurality of annotation question recommendation texts.

[0116] According to another aspect of the present disclosure, a text generation device is provided. Figure 8As shown, the text generation device 800 includes: a fourth acquisition unit 810, configured to acquire the target dialogue text and determine the target dialogue category of the target dialogue text; and a fourth large model input unit 820, configured to input the target dialogue text and the first prompt text corresponding to the target dialogue category into the large model in step to obtain the question recommendation text. The large model is trained using the above-mentioned device 700. It can be understood that the operations and effects of the units 810-820 in the device 800 can refer to the above description of steps S601-step S602.

[0117] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0118] According to an embodiment of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0119] refer to Fig. 9 , a block diagram of an electronic device 900 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0120] like Fig. 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0121] Multiple components in the electronic device 900 are connected to the I / O interface 905, including: an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. The input unit 906 can be any type of device that can input information to the electronic device 900. The input unit 906 can receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 907 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 908 can include but is not limited to a disk, an optical disk. The communication unit 909 allows the electronic device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0122] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods, processes, and / or processes described above. For example, in some embodiments, these methods, processes, and / or processes may be implemented as computer software programs, which are tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the methods, processes, and / or processes described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform these methods, processes, and / or processes in any other appropriate manner (e.g., by means of firmware).

[0123] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0125] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0127] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0128] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server for a distributed system, or a server combined with a blockchain.

[0129] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be performed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document is not limited here.

[0130] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A large model training method, comprising: Acquire a plurality of sample dialogue texts and a plurality of corresponding annotated question recommendation texts, wherein the plurality of sample dialogue texts belong to a plurality of dialogue categories; Determine a plurality of first prompt texts corresponding to the plurality of dialogue categories, wherein the first prompt text is used to instruct the first large model to generate a predicted question recommendation text that conforms to the dialogue category corresponding to the first prompt text; and The parameters of the first large model are adjusted using the multiple first prompt texts, the multiple sample dialogue texts and the multiple annotated question recommendation texts.

2. The method according to claim 1, wherein: Obtaining multiple sample dialogue texts and corresponding multiple labeled question recommendation texts includes: Determine a plurality of second prompt texts corresponding to the plurality of dialogue categories respectively, wherein the second prompt texts are used to instruct the trained second largest model to generate a labeling question recommendation text that conforms to the dialogue category corresponding to the second prompt text; and For a first sample dialogue text among the multiple sample dialogue texts, the first sample dialogue text and a second prompt text corresponding to a first dialogue category among the multiple dialogue categories are input into the second large model to obtain a first annotated question recommendation text corresponding to the first sample dialogue text, wherein the first sample dialogue text belongs to the first dialogue category.

3. The method according to claim 2, wherein: The first dialogue category includes a plurality of dialogue subcategories, the second prompt text corresponding to the first dialogue category includes a plurality of subcategory prompt texts corresponding to the plurality of dialogue subcategories, the subcategory prompt texts being used to instruct the second large model to generate a recommended text for annotated questions that conform to the corresponding dialogue subcategories, The first sample dialogue text belongs to the first dialogue subcategory among the multiple dialogue subcategories, and the first sample dialogue text and the subcategory prompt text corresponding to the first dialogue subcategory are input into the second large model to obtain the first annotation question recommendation text.

4. The method according to claim 3, further comprising: Acquire a first test dialogue text corresponding to the first dialogue subcategory; Inputting the first test dialogue text and the first prompt text corresponding to the first dialogue category into the adjusted first large model to obtain a first test question recommendation text; In response to determining that the first test question recommended text does not meet a first preset requirement, optimizing a subcategory prompt text corresponding to the first dialogue subcategory; Inputting the optimized subcategory prompt text and the first sample dialogue text into the second large model to obtain a second annotation question recommendation text; as well as The parameters of the first large model are readjusted using the first sample dialogue text, the first prompt text corresponding to the first dialogue category, and the second annotated question recommendation text.

5. The method according to claim 2, wherein: Obtaining multiple sample dialogue texts and corresponding multiple annotated question recommendation texts also includes: In response to determining that the first labeled question recommended text does not meet a second preset requirement, a second prompt text corresponding to the first dialogue category is optimized.

6. The method according to claim 5, wherein: Obtaining multiple sample dialogue texts and corresponding multiple annotated question recommendation texts also includes: determining a first batch of sample dialogue texts from the plurality of sample dialogue texts, the first batch of sample dialogue texts including the first sample dialogue text; In response to determining that the recommended texts for the labeled questions obtained by inputting the first batch of sample dialogue texts and the corresponding second prompt texts into the second large model all meet the second preset requirement, determining a second batch of sample dialogue texts from the plurality of sample dialogue texts, the second batch being larger than the first batch; and The second batch of sample dialogue texts and the corresponding second prompt texts are respectively input into the second large model to obtain the plurality of labeled question recommendation texts.

7. The method according to claim 2, wherein: The first large model has a smaller number of parameters than the second large model.

8. The method according to any one of claims 1 to 7, further comprising: Obtain multiple second test dialogue texts; Inputting the plurality of second test dialogue texts and the corresponding first prompt texts into the adjusted first large model respectively to obtain a plurality of second test question recommendation texts; as well as In response to determining that the plurality of second test question recommended texts do not satisfy a third preset requirement, parameters of the first large model are readjusted.

9. The method according to claim 8, wherein: The plurality of second test dialogue texts include a target test dialogue text, and the target test dialogue text corresponds to a test dialogue category in the plurality of dialogue categories, wherein, in response to determining that the plurality of second test question recommended texts do not meet a third preset requirement, re-adjusting the parameters of the first large model includes: In response to determining that a target test question recommendation text among the plurality of second test question recommendation texts does not meet the third preset requirement, optimizing a first prompt text corresponding to the test dialogue category, wherein the target test question recommendation text is obtained by inputting the target test dialogue text and the first prompt text corresponding to the test dialogue category into an adjusted first large model; and The optimized first prompt text is used to readjust the parameters of the first large model.

10. The method according to claim 8, wherein: In response to determining that the plurality of second test question recommended texts do not meet a third preset requirement, re-adjusting the parameters of the first large model includes: adjusting the ratio of the sample dialogue texts corresponding to the plurality of dialogue categories in the plurality of sample dialogue texts; and The parameters of the first large model are readjusted using the multiple sample dialogue texts after the adjusted ratio.

11. The method according to any one of claims 1 to 7, wherein: Using the plurality of first prompt texts, the plurality of sample dialogue texts, and the plurality of annotated question recommendation texts, adjusting the parameters of the first large model includes: For a second sample dialogue text among the plurality of sample dialogue texts, inputting the second sample dialogue text and a first prompt text corresponding to the second dialogue category into the first large model to obtain a predicted question recommendation text corresponding to the second sample dialogue text, wherein the second sample dialogue text belongs to the second dialogue category; and Adjust the parameters of the first large model based on the predicted question recommendation text and the annotated text recommendation text corresponding to the second sample dialogue text.

12. The method according to any one of claims 1 to 7, wherein: The multiple dialogue categories include quizzes, text creation, and small talk.

13. A text generation method, comprising: Acquire a target dialogue text, and determine a target dialogue category of the target dialogue text; as well as The target dialogue text and a first prompt text corresponding to the target dialogue category are input into a large model to obtain a question recommendation text, wherein the large model is trained using a method according to any one of claims 1-12.

14. A large model training device, comprising: A first acquisition unit is configured to acquire a plurality of sample dialogue texts and a plurality of corresponding annotated question recommendation texts, wherein the plurality of sample dialogue texts belong to a plurality of dialogue categories; a determining unit configured to determine a plurality of first prompt texts corresponding to the plurality of dialogue categories, wherein the first prompt texts are used to instruct the first large model to generate a predicted question recommendation text that conforms to the dialogue category corresponding to the first prompt text; and The first parameter adjustment unit is configured to adjust the parameters of the first large model using the multiple first prompt texts, the multiple sample dialogue texts and the multiple annotated question recommendation texts.

15. The device according to claim 14, wherein: The first acquiring unit includes: A first determining subunit is configured to determine a plurality of second prompt texts corresponding to the plurality of dialogue categories, wherein the second prompt texts are used to instruct the trained second largest model to generate a labeling question recommendation text that conforms to the dialogue category corresponding to the second prompt text; and The first large model input subunit is configured to input a first sample dialogue text among the multiple sample dialogue texts and a second prompt text corresponding to a first dialogue category among the multiple dialogue categories into the second large model to obtain a first annotated question recommendation text corresponding to the first sample dialogue text, wherein the first sample dialogue text belongs to the first dialogue category.

16. The device according to claim 15, wherein: The first dialogue category includes a plurality of dialogue subcategories, the second prompt text corresponding to the first dialogue category includes a plurality of subcategory prompt texts corresponding to the plurality of dialogue subcategories, the subcategory prompt texts being used to instruct the second large model to generate a labeling question recommendation text that conforms to the corresponding dialogue subcategory, The first sample dialogue text belongs to the first dialogue subcategory among the multiple dialogue subcategories, and the first sample dialogue text and the subcategory prompt text corresponding to the first dialogue subcategory are input into the second large model to obtain the first annotation question recommendation text.

17. The apparatus according to claim 16, further comprising: A second acquisition unit is configured to acquire a first test dialogue text corresponding to the first dialogue subcategory; A first large model input unit is configured to input the first test dialogue text and the first prompt text corresponding to the first dialogue category into the adjusted first large model to obtain a first test question recommendation text; an optimization unit, configured to, in response to determining that the first test question recommended text does not meet a first preset requirement, optimize a subcategory prompt text corresponding to the first dialogue subcategory; A second large model input unit is configured to input the optimized subcategory prompt text and the first sample dialogue text into the second large model to obtain a second annotation question recommendation text; as well as The second parameter adjustment unit is configured to readjust the parameters of the first large model by using the first sample dialogue text, the first prompt text corresponding to the first dialogue category, and the second annotated question recommendation text.

18. The device according to claim 15, wherein: The first acquiring unit includes: The first optimization subunit is configured to optimize a second prompt text corresponding to the first dialogue category in response to determining that the first labeled question recommended text does not meet a second preset requirement.

19. The device according to claim 18, wherein: The first acquiring unit includes: A second determining subunit is configured to determine a first batch of sample dialogue texts from the plurality of sample dialogue texts, the first batch of sample dialogue texts including the first sample dialogue text; A third determining subunit is configured to determine a second batch of sample dialogue texts from the plurality of sample dialogue texts in response to determining that the recommended texts for the labeled questions obtained after inputting the first batch of sample dialogue texts and the corresponding second prompt texts into the second large model all meet the second preset requirement, wherein the second batch is larger than the first batch; and The second large model input subunit is configured to input the second batch of sample dialogue texts and the corresponding second prompt texts into the second large model respectively, so as to obtain the plurality of annotated question recommendation texts.

20. The device according to claim 15, wherein: The first large model has a smaller number of parameters than the second large model.

21. The apparatus according to any one of claims 14 to 20, further comprising: A third acquisition unit is configured to acquire a plurality of second test dialogue texts; A third large model input unit is configured to input the plurality of second test dialogue texts and the corresponding first prompt texts into the adjusted first large model respectively to obtain a plurality of second test question recommendation texts; as well as The third parameter adjustment unit is configured to readjust the parameters of the first large model in response to determining that the plurality of second test question recommended texts do not meet a third preset requirement.

22. The device according to claim 21, wherein The plurality of second test dialogue texts include a target test dialogue text, and the target test dialogue text corresponds to a test dialogue category among the plurality of dialogue categories, wherein the third parameter adjustment unit includes: a second optimization subunit, configured to, in response to determining that a target test question recommendation text among the plurality of second test question recommendation texts does not meet the third preset requirement, optimize a first prompt text corresponding to the test dialogue category, wherein the target test question recommendation text is obtained by inputting the target test dialogue text and the first prompt text corresponding to the test dialogue category into an adjusted first large model; and The first parameter adjustment subunit is configured to readjust the parameters of the first large model using the optimized first prompt text.

23. The device according to claim 21, wherein The third parameter adjustment unit includes: a ratio adjustment subunit configured to adjust the ratios of the sample dialogue texts corresponding to the plurality of dialogue categories in the plurality of sample dialogue texts; and The second parameter adjustment sub-unit is configured to readjust the parameters of the first large model using the multiple sample dialogue texts after the adjustment ratio.

24. The device according to any one of claims 14 to 20, wherein: The first parameter adjustment unit includes: A third large model input subunit is configured to input a second sample dialogue text among the plurality of sample dialogue texts and a first prompt text corresponding to the second dialogue category into the first large model to obtain a predicted question recommendation text corresponding to the second sample dialogue text, wherein the second sample dialogue text belongs to the second dialogue category; and The third parameter adjustment sub-unit is configured to adjust the parameters of the first large model based on the predicted question recommendation text and the annotated text recommendation text corresponding to the second sample dialogue text.

25. The device according to any one of claims 14 to 20, wherein: The multiple dialogue categories include quizzes, text creation, and small talk.

26. A text generation device, comprising: a fourth acquisition unit, configured to acquire a target dialogue text and determine a target dialogue category of the target dialogue text; as well as The fourth large model input unit is configured to input the target dialogue text and the first prompt text corresponding to the target dialogue category into the large model in step to obtain the question recommendation text, wherein the large model is trained using the device according to any one of claims 14-25.

27. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; wherein The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

28. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

29. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images