Information processing method and device based on large model and intelligent agent

Abstract

The invention provides an information processing method and device based on a large model and an intelligent agent, and relates to the technical field of artificial intelligence, in particular to the technical fields of natural language processing technologies, deep learning, large language models, man-machine interaction and the like. The sample generation method based on the large model comprises the following steps: calling the large model obtained by the last round of training, and processing target reference information used for the current round of training according to prompt information to obtain at least one piece of question information and a plurality of pieces of candidate answer information of each piece of question information; and for each piece of question information, according to the question information and at least two pieces of target answer information in the multiple pieces of candidate answer information, generating a training sample for the current round of training.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to natural language processing technology, deep learning, large language models, human-computer interaction and other technical fields, and more specifically to information processing methods, devices and intelligent agents based on large models. Background Art

[0002] With the development of deep learning technology, large language models (LLMs) have emerged. Large language models refer to language models with a large number of parameters and pre-trained data obtained by training on large-scale text data using deep learning technology. Large language models can be used for various natural language processing tasks such as dialogue generation and text generation to obtain natural, fluent and accurate text output. Summary of the invention

[0003] The present invention provides a large model-based information processing method, device and intelligent agent.

[0004] According to one aspect of the present disclosure, a sample generation method based on a large model is provided, comprising: calling the large model obtained in the previous round of training, and processing the target reference information used for the current round of training according to the prompt information to obtain at least one question information and multiple candidate answer information for each of the above question information; and, for each of the above question information, generating a training sample for the current round of training according to the above question information and at least two target answer information among the above multiple candidate answer information.

[0005] According to one aspect of the present disclosure, a training method for a large model is provided, comprising: inputting at least one question information in a training sample for a current round of training into a large model obtained by a previous round of training, respectively, to obtain output information for each of the above question information, wherein the above training samples for the current round of training are obtained by using a sample generation method based on a large model, and the above training samples for the current round of training also include at least two target answer information for each of the above question information; and, training the large model obtained by the previous round of training according to the above output information and the above at least two target answer information for each of the above question information, to obtain a large model obtained by the current round of training.

[0006] According to one aspect of the present disclosure, there is provided an information processing method based on a large model, comprising: obtaining information to be processed; and inputting the information to be processed into a target large model to obtain an information processing result, wherein the target large model is obtained using a large model training method.

[0007] According to another aspect of the present disclosure, a sample generation device based on a large model is provided, including: a processing module, which is used to call the large model obtained in the previous round of training, and process the target reference information used for the current round of training according to prompt information to obtain at least one question information and multiple candidate answer information for each of the above question information; and a generation module, which is used to generate a training sample for the current round of training for each of the above question information according to the above question information and at least two target answer information among the above multiple candidate answer information.

[0008] According to another aspect of the present disclosure, a training device for a large model is provided, comprising: a first input module, used to input at least one question information in the training samples for the current round of training into the large model obtained by the previous round of training, to obtain output information of each of the above question information, wherein the above training samples for the current round of training are obtained by using a sample generation device based on the large model, and the above training samples for the current round of training also include at least two target answer information of each of the above question information; and a first training module, used to train the above large model trained by the previous round according to the above output information of each of the above question information and the above at least two target answer information, to obtain the large model obtained by the current round of training.

[0009] According to another aspect of the present disclosure, there is provided an information processing device based on a large model, comprising: a second acquisition module, used to acquire information to be processed; and a second input module, used to input the above-mentioned information to be processed into a target large model to obtain an information processing result, wherein the above-mentioned target large model is obtained using a large model training device.

[0010] According to another aspect of the present disclosure, an intelligent agent is provided, including: an input module for receiving input information; a processing module for determining a target task based on the input information received by the input module, determining a large model based on the target task, and obtaining output information by calling the large model to execute an information processing method based on the large model; and an output module for outputting the output information obtained by the processing module.

[0011] According to another aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0012] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0013] According to another aspect of the present disclosure, a computer program product is provided, including a computer program or instructions, and the steps of the above method are implemented when the computer program or instructions are executed by a processor.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0016] Figure 1 The system architecture to which the sample generation method based on a large model, the training method based on a large model, and the information processing method based on a large model according to the embodiments of the present disclosure can be applied is schematically shown;

[0017] Figure 2 The flowchart of the sample generation method based on the large model according to the embodiment of the present disclosure is schematically shown;

[0018] Figure 3 An example schematic diagram schematically shows a process of obtaining target reference information of the current round according to an embodiment of the present disclosure;

[0019] Figure 4 An example schematic diagram schematically shows a process of generating question information and candidate answer information according to an embodiment of the present disclosure;

[0020] Figure 5 An example schematic diagram schematically shows a training sample generation process according to an embodiment of the present disclosure;

[0021] Figure 6 A flowchart of a large model training method according to an embodiment of the present disclosure is schematically shown;

[0022] Figure 7 An example schematic diagram of the first round of large model training process according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 8 An example schematic diagram schematically shows a large model training process of the current round according to an embodiment of the present disclosure;

[0024] Fig. 9 An example schematic diagram schematically shows a process of obtaining a target macro model according to an embodiment of the present disclosure;

[0025] Fig.10 A flowchart of a large model-based information processing method according to an embodiment of the present disclosure is schematically shown;

[0026] Fig.11 A block diagram of a sample generation device based on a large model according to an embodiment of the present disclosure is schematically shown;

[0027] Fig.12 A block diagram of a large model training device according to an embodiment of the present disclosure is schematically shown;

[0028] Fig.13 A block diagram of an information processing device based on a large model according to an embodiment of the present disclosure is schematically shown;

[0029] Fig.14 A block diagram schematically shows a structure of an intelligent agent of a large model according to an embodiment of the present disclosure; and

[0030] Fig.15 A block diagram of an electronic device suitable for implementing a sample generation method based on a large model, a training method of a large model, and an information processing method based on a large model according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0031] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0032] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0033] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0034] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0035] With the development of deep learning technology, large language models (LLM) have emerged. At present, the research on large language models mainly focuses on using closed-source large language models to complete the instructions proposed by users. However, in some application scenarios, due to the low operating efficiency of closed-source large language models and their inability to handle tasks in specific business areas, open-source large language models are needed to train and complete tasks.

[0036] In one example, the training methods for an open source large language model may include a training method based on reinforcement learning from human feedback (RLHF) and a training method based on direct preference optimization (DPO). However, the training method based on reinforcement learning from human feedback relies on a fixed reward model, so if the effect of the reward model is poor, it will affect the training effect of the model; the training method based on direct preference optimization will be limited by the quantity and quality of data.

[0037] Therefore, how to enable the model to gradually improve its instruction-following and evaluation capabilities and achieve self-optimization is an urgent problem to be solved.

[0038] To this end, the embodiment of the present disclosure proposes a sample generation scheme based on a large model. For example, by calling the large model obtained through the previous round of training, according to the prompt information, the target reference information used for the current round of training is processed to obtain at least one question information and multiple candidate answer information for each question information; and for each question information, according to the question information and at least two target answer information among the multiple candidate answer information, a training sample for the current round of training is generated.

[0039] According to the embodiments of the present disclosure, by calling the large model obtained from the previous round of training, processing the target reference information according to the prompt information, and automatically generating question information and candidate answer information, the efficiency of generating question information and candidate answer information is improved, and the time cost of manual intervention is reduced. On this basis, by generating multiple candidate answer information for each question information, and selecting at least two target answer information from them to automatically generate training samples for the current round of training, the diversity and accuracy of the training samples are guaranteed.

[0040] In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0041] In the technical solution of the present invention, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0042] Figure 1 The schematic diagram shows the system architecture to which the sample generation method based on the large model, the training method based on the large model, and the information processing method based on the large model can be applied according to the embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0043] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0044] The user may use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0045] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0046] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0047] It should be noted that the sample generation method based on the large model, the training method based on the large model, and the information processing method based on the large model provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the sample generation device based on the large model, the training device based on the large model, and the information processing device based on the large model provided in the embodiments of the present disclosure can generally be set in the server 105.

[0048] The sample generation method based on the large model, the training method based on the large model, and the information processing method based on the large model provided in the embodiments of the present disclosure may also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the sample generation device based on the large model, the training device based on the large model, and the information processing device based on the large model provided in the embodiments of the present disclosure may also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0049] Alternatively, the large model-based sample generation method, large model training method, and large model-based information processing method provided in the embodiments of the present disclosure may also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the large model-based sample generation device, large model training device, and large model-based information processing device provided in the embodiments of the present disclosure may also be set in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0050] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0051] It should be noted that the sequence numbers of the operations in the following method are only used as representations of the operations for the purpose of description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed completely in the order shown.

[0052] The above describes the system architecture of the sample generation method based on the large model, the training method based on the large model, and the information processing method based on the large model provided by the present disclosure. Figure 2As an example, the sample generation method based on a large model disclosed in the present invention is further described.

[0053] Figure 2 The flowchart of the sample generation method based on the large model according to the embodiment of the present disclosure is schematically shown.

[0054] like Figure 2 As shown, the large model-based sample generation method 200 includes operations S210 to S220.

[0055] In operation S210, by calling the large model obtained in the previous round of training, the target reference information used for the current round of training is processed according to the prompt information to obtain at least one question information and multiple candidate answer information for each question information.

[0056] In operation S220, for each question information, a training sample for current round of training is generated according to the question information and at least two target answer information among a plurality of candidate answer information.

[0057] The large model-based sample generation method provided in the present disclosure can be applied to a large language model trained based on massive text data, but is not limited to this. The large model-based sample generation method provided in the present disclosure can also be applied to a multimodal large language model trained by combining multiple modal information such as text, images, videos, and audio.

[0058] The training process of the large model is an iterative process of multiple rounds. For each round in the iterative process, the target reference information for the current round of training is selected, and the training samples for the current round of training are generated, so that the training samples for the current round can be used to train the large model obtained by the previous round of training, thereby obtaining the large model obtained by the current round of training.

[0059] For the current round, target reference information for the current round can be determined. The target reference information may include multiple initial question information and actual answer information for each initial question information. The initial question information refers to the query, question or demand raised by the user. The initial question information may be in text form or in other modalities such as voice and image. The actual answer information refers to the correct answer to the initial question information, which may be a direct answer, explanation, or the result of performing a task.

[0060] Prompt information can be instruction information used to guide the large model to perform specific tasks or generate output data in a specific format. Prompt information can be configured according to actual business needs and is not limited here. For example, prompt information may include at least one of the following: question generation instructions, answer generation instructions, and instruction evaluation instructions. Question generation instructions can be used to provide constraints for the generation of question information, and can also specify the format, length, and style of question information. Answer generation instructions can be used to provide constraints for the generation of answer information, and can also set scenarios and provide thinking frameworks for the generation of answer information, and can also specify the format, length, and style of answer information output. Instruction evaluation instructions can be used to evaluate answer information in different dimensions. For example, multiple dimensions such as the content, structure, and language style of the answer information can be evaluated.

[0061] In one example, when the prompt information includes a question generation instruction, the question generation instruction and the target reference information can be input into the big model to obtain new question information. In another example, when the prompt information includes an answer generation instruction, the answer generation instruction and the question information can be input into the big model to obtain new candidate answer information. For the answer generation process, multiple new candidate answer information can be obtained by inputting the answer generation instruction and the question information into the big model multiple times.

[0062] For each question information, after obtaining multiple candidate answer information, at least two target answer information can be determined from the multiple candidate answer information. The method for obtaining the target answer information can be configured according to actual business needs and is not limited here. For example, a preset rule for indicating the determination of the target answer information can be set, and the multiple candidate answer information can be evaluated respectively based on the preset rule, so as to determine at least two target answer information from the multiple candidate answer information.

[0063] After obtaining at least two target answer information for each question information, a training sample for the current round of training can be generated according to each question information and the at least two target answer information for each question information. The training sample for the current round of training can be used to subsequently train the large model obtained by the previous round of training to obtain the large model obtained by the current round of training.

[0064] According to the embodiments of the present disclosure, by calling the large model obtained from the previous round of training, processing the target reference information according to the prompt information, and automatically generating question information and candidate answer information, the efficiency of generating question information and candidate answer information is improved, and the time cost of manual intervention is reduced. On this basis, by generating multiple candidate answer information for each question information, and selecting at least two target answer information from them to automatically generate training samples for the current round of training, the diversity and accuracy of the training samples are guaranteed.

[0065] The above describes the sample generation process based on the large model provided by the present disclosure. In the embodiment of the present disclosure, the target reference information can be obtained from the initial reference information. Figure 3 The process of obtaining target reference information is further explained.

[0066] Figure 3 An example schematic diagram of a target reference information acquisition process of the current round according to an embodiment of the present disclosure is schematically shown.

[0067] like Figure 3 As shown, in 300, initial reference information 301 may be obtained. The initial reference information 301 may include a second number of initial question information 3011 and actual answer information 3012 of each initial question information 3011. The actual answer information 3012 refers to the correct answer of each initial question information 3011.

[0068] In one example, the method for obtaining the initial reference information 301 can be configured according to actual business needs and is not limited here. For example, the method for obtaining the initial reference information 301 can include at least one of the following: an obtaining method based on querying a knowledge base, an obtaining method based on an expert system, an obtaining method based on retrieval-based question-answering, an obtaining method based on information extraction, and an obtaining method based on manual annotation, etc.

[0069] After obtaining the initial reference information 301, a first number of initial question information 3011 and actual answer information 3012 of each initial question information 3011 can be determined in the initial reference information 301 according to a preset ratio 302 to obtain target reference information 303 for the current round of training.

[0070] In one example, the preset ratio 302 may represent the relative relationship between the first number and the second number, and the preset ratio 302 may be the ratio of the first number to the second number, where the second number is greater than the first number. For example, if the first number is 4 and the second number is 2, then the preset ratio is 2:1, indicating that 2 sets of initial question information 3011 and corresponding actual answer information 3012 are randomly sampled from 4 sets of initial question information 3011 and corresponding actual answer information 3012 as target reference information for the current round of training.

[0071] According to an embodiment of the present disclosure, by selecting part of the question information and part of the actual answer information from the initial reference information according to a preset ratio for the current round of training, it is possible to control the amount of training data while ensuring sample diversity, thereby avoiding the overfitting problem that may be caused by processing too many samples at one time.

[0072] The above describes the process of obtaining target reference information. In the embodiment of the present disclosure, the target reference information can be input into the large model obtained by the previous round of training to generate question information and candidate answer information. Figure 4 The generation process of question information and candidate answer information is further explained.

[0073] Figure 4 An example schematic diagram of a process for generating question information and candidate answer information according to an embodiment of the present disclosure is schematically shown.

[0074] like Figure 4 As shown in 400 , the target reference information may include a first number of initial question information 401 and actual answer information 402 of each initial question information 401 .

[0075] The prompt information may include a question generation instruction 403. The question generation instruction 403 may be one or more sets of instructions, commands or constraints to instruct the large model on how to generate question information. For each initial question information 401, in response to obtaining the question generation instruction 403 in the prompt information, the question generation instruction 403, the initial question information 401 and the actual answer information 402 may be input into the large model 404, and at least one question information 405 may be obtained by using the imitation capability of the large model 404.

[0076] In one example, the large model 404 may perform semantic analysis and grammatical analysis on the initial question information 401 to capture the intent of the initial question information 401 . On this basis, the question information 405 may be generated based on the question generation instruction 403 and the intent of the initial question information 401 .

[0077] The prompt information may also include an answer generation instruction 406. The answer generation instruction 406 may be one or more groups of instructions, commands or constraints to instruct the large model on how to generate candidate answer information for the question information. For each question information 405, in response to obtaining the answer generation instruction 406 in the prompt information, the answer generation instruction 406 and the question information 405 are input into the large model 404, and the reasoning ability of the large model 404 is used to obtain multiple candidate answer information 407 for each question information 405.

[0078] In one example, the big model 404 can perform semantic analysis and grammatical analysis on the initial question information 401 to capture the intent of the initial question information 401, the type of candidate answer information 407 required, the range of candidate answer information 407, etc. On this basis, the back-string answer information 407 can be generated based on the answer generation instruction 406 and the intent of the initial question information 401, the type of candidate answer information 407 required, and the range of candidate answer information 407.

[0079] According to an embodiment of the present disclosure, question information is automatically generated by inputting question generation instructions, initial question information and actual answer information into a large model. Similarly, multiple candidate answer information is automatically generated by inputting answer generation instructions and question information into the large model, thereby reducing the workload of manually designing question information and answer information, improving the degree of automation of question generation and answer generation, and ensuring the diversity and richness of question information and candidate answer information.

[0080] The above describes the process of generating question information and candidate answer information. In the embodiment of the present disclosure, a training sample can be generated based on the question information and at least two candidate answer information among the multiple candidate answer information. Figure 5 The generation process of training samples is further explained.

[0081] Figure 5 An example schematic diagram of a training sample generation process according to an embodiment of the present disclosure is schematically shown.

[0082] like Figure 5 As shown in 500, the multiple candidate answer information may include candidate answer information 5011, candidate answer information 5012, ..., candidate answer information 501M, where M is a positive integer greater than 1.

[0083] The prompt information may include a quality assessment instruction 502. The quality assessment instruction 502 may be one or more groups of instructions, commands, etc. The correctness, completeness, logic, timeliness, and output format of the candidate answer information may be evaluated, and corresponding evaluation values ​​may be generated. After obtaining a plurality of candidate answer information, in response to obtaining the quality assessment instruction 502 in the prompt information, the quality assessment instruction 502 and each candidate answer information are respectively input into the large model 503 to obtain the evaluation values ​​of the plurality of candidate answer information. For example, the evaluation value 5041 of the candidate answer information 5011, the evaluation value 5042 of the candidate answer information 5012, and the evaluation value 504M of the candidate answer information 504M.

[0084] According to the embodiments of the present disclosure, by inputting the quality assessment instructions and the candidate answer information into the big model, the quality assessment of multiple candidate answer information can be automatically performed, which reduces the subjectivity and tedious operations of manual assessment and improves work efficiency. The generated evaluation value provides an objective quantitative standard for each candidate answer information, so that the big model can judge the quality of the candidate answer according to the evaluation value, which helps to ensure that the training samples used for training have higher accuracy and reliability.

[0085] After obtaining the evaluation values ​​of the multiple candidate answer information, at least two target answer information 507 can be determined from the multiple candidate answer information based on the preset rules. The preset rules can be configured according to actual business needs and are not limited here. For example, the preset rules can be used to screen the multiple candidate answer information based on the evaluation values.

[0086] In one example, for each question information 508, the evaluation values ​​of the multiple candidate answer information corresponding to the question information 508 can be sorted to obtain a plurality of sorted evaluation values. For the sorted evaluation values, at least one candidate answer information corresponding to the highest evaluation value 505 and at least one candidate answer information corresponding to the lowest evaluation value 506 can be determined as target answer information 507. The candidate answer information corresponding to the highest evaluation value 505 can refer to high-quality answer information. The candidate answer information corresponding to the lowest evaluation value 506 can refer to low-quality answer information.

[0087] According to an embodiment of the present disclosure, by selecting candidate answer information corresponding to the highest evaluation value and candidate answer information corresponding to the lowest evaluation value as target answer information, the quality and diversity of training samples are improved, and the generalization ability of the trained large model and the ability to distinguish between high-quality answers and low-quality answers are enhanced.

[0088] After obtaining the target answer information 507 of each question information 508, for each question information 508, a sample pair 509 of a triple of the question information 508 can be generated according to the question information 508, the candidate answer information corresponding to the highest evaluation value 505, and the candidate answer information corresponding to the lowest evaluation value 506. The sample pair 509 can also be called preference data. On this basis, a training sample 510 for the current round of training can be generated according to at least one question information 508 and the sample pair 509 of each question information 508.

[0089] According to an embodiment of the present disclosure, by generating a sample pair containing candidate answer information corresponding to the highest evaluation value and candidate answer information corresponding to the lowest evaluation value for each question information, it is possible to ensure that the training samples not only contain high-quality positive data, but also introduce low-quality negative data, thereby helping the model to better learn how to distinguish answers of different qualities and improve its understanding of and ability to respond to problems.

[0090] The above are merely exemplary embodiments, but are not limited thereto, and may also include other large model-based sample generation methods known in the art, as long as the diversity and accuracy of the training samples can be guaranteed.

[0091] The above describes the sample generation method based on the large model provided by the present disclosure. Figure 6 For example, the training method of the large model disclosed in the present invention is further explained.

[0092] Figure 6 A flowchart of a large model training method according to an embodiment of the present disclosure is schematically shown.

[0093] like Figure 6 As shown, the large model training method 600 includes operations S610 to S620.

[0094] In operation S610, at least one question information in the training samples for the current round of training is respectively input into the large model obtained through the previous round of training to obtain output information for each question information, wherein the training samples for the current round of training are obtained by using a sample generation method based on the large model, and the training samples for the current round of training also include at least two target answer information for each question information.

[0095] In operation S620, the large model obtained by the previous round of training is trained according to the output information of each question information and at least two target answer information, so as to obtain the large model obtained by the current round of training.

[0096] After obtaining the training samples for the current round of training, for each question information in the training samples, the question information can be input into the large model obtained by the previous round of training to obtain the output information of each question information. On this basis, the difference between the output information and at least two target answer information can be determined, and the difference can be quantified by the loss function to obtain the target loss function value, so as to adjust the model parameters of the large model obtained by the previous round of training based on the target loss function value to obtain the large model obtained by the current round of training.

[0097] In the process of iterative training of the big model, the big model can obtain instruction-following ability, generation ability and evaluation ability. Instruction-following ability means that the big model can output high-quality and helpful answer information according to user requirements. Generation ability means that the big model can generate new sample data for training. Evaluation ability means that the big model can evaluate the quality of the generated answer information. The above-mentioned abilities help to obtain a more powerful target big model in the entire iterative process.

[0098] According to an embodiment of the present disclosure, by utilizing the training samples generated for the current round in the self-guidance stage to train the large model obtained through the previous round of training, the model can gradually improve its instruction-following and evaluation capabilities, perform self-improvement and adjustment, gradually improve its accuracy and adaptability, and thereby effectively improve the model performance.

[0099] The training process of the large model provided by the present disclosure is described above. In the embodiment of the present disclosure, the large model of the first round can be trained using the initial reference information. Figure 7 The training process of the first round of large model is further explained.

[0100] Figure 7 An example schematic diagram of the first round of large model training process according to an embodiment of the present disclosure is schematically shown.

[0101] like Figure 7 As shown, in 700 , initial reference information may be obtained when the current round is equal to 1. The initial reference information may refer to seed data, and the initial reference information may include a second number of initial question information 701 and actual answer information 702 of each initial question information 701 .

[0102] For each initial question information 701, the initial question information 701 can be input into the initial large model 703 to obtain predicted question information 704. The initial large model can refer to an untrained large model. On this basis, the model parameters of the initial large model 703 can be adjusted according to the difference between the predicted question information 704 and the actual answer information 702 corresponding to the initial question information 701, so as to obtain a large model obtained through the first round of training with basic instruction following ability.

[0103] According to an embodiment of the present disclosure, when the current round is equal to 1, by obtaining initial reference information and using the initial reference information to perform preliminary training on the initial large model, the large model can have basic instruction-following capabilities, thereby ensuring the rationality and effectiveness of the subsequent training process.

[0104] The training process of the large model in the first round is described above. In the embodiment of the present disclosure, the large model obtained by the current round of training can be obtained by using the training samples used for the current round of training to train the large model obtained by the previous round of training. Figure 8 The training process of the large model obtained through the current round of training is further explained.

[0105] Figure 8 An example schematic diagram of the current round of large model training process according to an embodiment of the present disclosure is schematically shown.

[0106] like Figure 8 As shown, in 800 , for each question information 801 , at least two target answer information may include answer information 802 corresponding to the highest evaluation value and answer information 803 corresponding to the lowest evaluation value.

[0107] The question information 801 is input into the large model 804 obtained by the previous round of training to obtain output information 805. According to the output information 804 and the answer information 802 corresponding to the highest evaluation value, based on the gap between the output information 804 output by the preset loss function quantification model and the high-quality answer information, a first loss function value 806 is obtained. According to the output information 804 and the answer information 803 corresponding to the lowest evaluation value, based on the gap between the output information 804 output by the preset loss function quantification model and the low-quality answer information, a second loss function value 807 is obtained. On this basis, the target loss function value 808 can be determined according to the first loss function value 806 and the second loss function value 807.

[0108] The goal of the preset loss function is to make the output information 804 and the answer information 802 corresponding to the highest evaluation value close to each other, even if the distance between the output information 804 and the answer information 802 corresponding to the highest evaluation value is smaller; to make the output information 804 and the answer information 803 corresponding to the lowest evaluation value farther away from each other, even if the distance between the output information 804 and the answer information 803 corresponding to the lowest evaluation value is larger. The preset loss function can be configured according to actual business needs and is not limited here. For example, the preset loss function can include at least one of the following: Cross-Entropy Loss, Contrastive Loss, Triplet Loss, etc.

[0109] After obtaining the target loss function value 808, the model parameters of the large model 804 obtained by the previous round of training can be adjusted based on the target loss function value 808 to obtain the large model 809 obtained by the current round of training.

[0110] According to an embodiment of the present disclosure, by determining the first loss function value and the second loss function value based on the output information and the answer information corresponding to the highest evaluation value and the lowest evaluation value, and determining the target loss function value based on these loss function values, the quality of the model output can be measured more accurately, not only considering the degree of match with high-quality answers, but also evaluating the gap with low-quality answers, thereby ensuring that the model can be effectively adjusted during the training process to improve accuracy and comprehensiveness.

[0111] The training process of the large model obtained by the current round of training is described above. In the embodiment of the present disclosure, the large model obtained by the current round of training can also be tested so that the iteration can be stopped when the test result meets the predetermined condition to obtain the target large model. Fig. 9 The process of obtaining the target large model is further explained.

[0112] Fig. 9An example schematic diagram of a process for obtaining a target macro model according to an embodiment of the present disclosure is schematically shown.

[0113] like Fig. 9 As shown, in 900, after obtaining the large model 902 obtained by the current round of training, the large model 902 obtained by the current round of training can be tested using the test sample 901 to obtain the test result 903. The test sample may include multiple test question information and actual answer information corresponding to each test question information. For example, the test question information can be input into the large model 902 obtained by the current round of training to obtain output information, and the test result 903 is determined based on the output information and the actual answer information. The test result 903 can be used to characterize the performance evaluation of the large model.

[0114] After obtaining the test result 903, operation S910 can be performed. In operation S910, does the test result 903 meet the predetermined conditions? The predetermined conditions can be configured according to actual business needs and are not limited here. For example, the preset conditions may include at least one of the following: accuracy, precision, recall, F1 score, AUC (area under the curve), etc. If not, the next round of iterative training can be continued. If so, the large model 902 obtained by the current round of training can be determined as the target large model 904.

[0115] According to the embodiments of the present disclosure, by using test samples to test the large model obtained by the current round of training, obtaining test results, and evaluating the test results according to predetermined conditions, it is possible to ensure that the model is determined as the target large model only after it reaches the set performance standard. This can accurately determine the timing of termination of the iterative training process, avoid ineffective training for too long, and improve training efficiency.

[0116] The above are only exemplary embodiments, but are not limited thereto, and may also include other large model training methods known in the art, as long as they can achieve self-improvement and adjustment of the large model.

[0117] The above describes the training method of the large model provided by the present disclosure. Fig.10 As an example, the information processing method based on a large model disclosed in the present invention is further described.

[0118] Fig.10 The flowchart of the information processing method based on the large model according to the embodiment of the present disclosure is schematically shown.

[0119] like Fig.10 As shown, the large model-based information processing method 1000 includes operations S1010 to S1020.

[0120] In operation S1010, information to be processed is obtained.

[0121] In operation S1020, the information to be processed is input into the target large model to obtain an information processing result, wherein the target large model is obtained by using a large model training method.

[0122] After obtaining the information to be processed, the information to be processed can be input into the target large model so that the target large model outputs an information processing result for answering the information to be processed.

[0123] In one example, the information to be processed by the target large model belongs to the same business domain as the initial reference information originally used to train the target large model, thereby ensuring that the target large model is learned and applied in a similar context and background, thereby generating output that meets the needs of a specific business domain. The specific business domain can be configured according to actual business needs and is not limited here. For example, the business domain may include at least one of the following: office field, financial field, medical field, legal field, education field, and e-commerce field, etc.

[0124] According to the embodiments of the present disclosure, since the target big model is trained based on relevant training data in a specific business field, the target big model can demonstrate stronger understanding and accuracy when processing the information to be processed in the same business field, ensuring that the target big model can handle specific tasks in a more targeted manner, thereby improving the accuracy of the information processing results.

[0125] The above are merely exemplary embodiments, but are not limited thereto, and may also include other large model-based information processing methods known in the art, as long as the accuracy of the information processing results can be improved.

[0126] Fig.11 A block diagram of a sample generation device based on a large model according to an embodiment of the present disclosure is schematically shown.

[0127] like Fig.11 As shown, the large model-based sample generation apparatus 1100 may include a processing module 1110 and a generation module 1120 .

[0128] The processing module 1110 is used to process the target reference information used for the current round of training by calling the large model obtained from the previous round of training according to the prompt information, so as to obtain at least one question information and multiple candidate answer information for each question information.

[0129] The generating module 1120 is used to generate, for each question information, a training sample for the current round of training according to the question information and at least two target answer information among the multiple candidate answer information.

[0130] According to an embodiment of the present disclosure, the target reference information includes a first number of initial question information and actual answer information of each initial question information.

[0131] According to an embodiment of the present disclosure, the processing module 1110 may include a first processing unit and a second processing unit.

[0132] The first processing unit is used for inputting the question generation instruction, the initial question information and the actual answer information into the large model for each initial question information in response to the question generation instruction obtained in the prompt information, so as to obtain at least one question information.

[0133] The second processing unit is used for inputting the answer generation instruction and the question information into the large model for each question information in response to the answer generation instruction obtained in the prompt information, so as to obtain multiple candidate answer information for each question information.

[0134] According to an embodiment of the present disclosure, the generating module 1120 may include a first determining unit and a generating unit.

[0135] The first determining unit is used to determine at least two target answer information from a plurality of candidate answer information based on a preset rule.

[0136] The generating unit is used to generate training samples for the current round of training according to the question information and at least two target answer information.

[0137] According to an embodiment of the present disclosure, the generating module 1120 may further include an evaluating unit.

[0138] The evaluation unit is used to respond to the quality evaluation instruction obtained in the prompt information, input the quality evaluation quality and the candidate answer information into the large model, and obtain the evaluation value of each of the multiple candidate answer information.

[0139] According to an embodiment of the present disclosure, a preset rule is used to screen a plurality of candidate answer information based on an evaluation value.

[0140] According to an embodiment of the present disclosure, the first determining unit may include a first determining subunit and a second determining subunit.

[0141] The first determining subunit is configured to determine at least one candidate answer information corresponding to a highest evaluation value among the multiple evaluation values ​​as target answer information.

[0142] The second determining subunit is configured to determine at least one candidate answer information corresponding to the lowest evaluation value among the multiple evaluation values ​​as the target answer information.

[0143] According to an embodiment of the present disclosure, the generation unit may include a first generation subunit and a second generation subunit.

[0144] The first generating subunit is used to generate, for each piece of question information, a sample pair of question information according to the question information, the candidate answer information corresponding to the highest evaluation value, and the candidate answer information corresponding to the lowest evaluation value.

[0145] The second generating subunit is used to generate training samples for the current round of training according to at least one question information and a sample pair of each question information.

[0146] According to an embodiment of the present disclosure, the target reference information used for the current round of training is obtained in the following manner.

[0147] According to a preset ratio, a first number of initial question information and actual answer information of each initial question information are determined in the initial reference information to obtain target reference information for the current round of training. The initial reference information includes a second number of initial question information and actual answer information of each initial question information, the preset ratio is the ratio of the first number to the second number, and the second number is greater than the first number.

[0148] Fig.12 A block diagram of a large model training device according to an embodiment of the present disclosure is schematically shown.

[0149] like Fig.12 As shown, the large model training device 1200 may include a first input module 1210 and a first training module 1220 .

[0150] The first input module 1210 is used to input at least one question information in the training samples for the current round of training into the large model obtained through the previous round of training, and obtain output information for each question information, wherein the training samples for the current round of training are obtained by using the sample generation device 1100 based on the large model, and the training samples for the current round of training also include at least two target answer information for each question information.

[0151] The first training module 1220 is used to train the large model obtained by the previous round of training according to the output information of each question information and at least two target answer information, so as to obtain the large model obtained by the current round of training.

[0152] According to an embodiment of the present disclosure, the at least two target answer information include answer information corresponding to the highest evaluation value and answer information corresponding to the lowest evaluation value.

[0153] According to an embodiment of the present disclosure, when the current round is greater than 1, the first training module 1220 may include a second determining unit and an adjusting unit.

[0154] The second determination unit is used to determine the target loss function value for each question information based on a first loss function value determined based on the output information and the answer information corresponding to the highest evaluation value, and a second loss function value determined based on the output information and the answer information corresponding to the lowest evaluation value.

[0155] The adjustment unit is used to adjust the model parameters of the large model obtained by the previous round of training based on the target loss function value to obtain the large model obtained by the current round of training.

[0156] According to an embodiment of the present disclosure, the large model training device 1200 may further include a testing module and a determining module.

[0157] The test module is used to use the test samples to test the large model obtained through the current round of training to obtain the test results.

[0158] The determination module is used to determine the large model obtained through the current round of training as the target large model in response to the test result satisfying the predetermined condition.

[0159] According to an embodiment of the present disclosure, when the current round is equal to 1, the training device 1200 for the large model may further include a first acquisition module and a second training module.

[0160] The first acquisition module is used to acquire initial reference information, wherein the initial reference information includes a second number of initial question information and actual answer information of each initial question information.

[0161] The second training module is used to train the initial large model using the initial reference information to obtain the large model obtained through the first round of training.

[0162] Fig.13 A block diagram of a large model-based information processing device according to an embodiment of the present disclosure is schematically shown.

[0163] like Fig.13 As shown, the large model-based information processing device 1300 may include a second acquisition module 1310 and a second input module 1320 .

[0164] The second acquisition module 1310 is used to acquire information to be processed.

[0165] The second input module 1320 is used to input the information to be processed into the target large model to obtain the information processing result, wherein the target large model is obtained by using the large model training device 1200.

[0166] According to an embodiment of the present disclosure, the information to be processed and the initial reference information used to train the target large model belong to the same business field.

[0167] Fig.14 The structural block diagram of the intelligent agent of the large model according to the embodiment of the present disclosure is schematically shown.

[0168] In the embodiments of the present disclosure, inspired by the von Neumann structure in modern computer theory, such as Fig.14 As shown, the AI ​​agent 1400 may include: an input module 1410, a processing module 1420 and an output module 1430.

[0169] The input module 1410 can be used to receive or perceive input information such as queries, requests, instructions, signals or data from the outside world (such as users or external environment), and convert it into a format that can be understood and processed by the AI ​​agent 1400. The input module 1410 is the primary link for the AI ​​agent 1400 to interact with the outside world. It enables the AI ​​agent 1400 to efficiently and accurately obtain the necessary "sensory" information from the outside world and respond to this information.

[0170] In an example, the input information may be the target reference information described above.

[0171] The processing module 1420 can be used to determine the target task based on the input information received by the input module 1410, determine the large model based on the target task, obtain output information by calling the large model to execute the method described above, and pass the output information to the output module 1430.

[0172] In an example, the processing module 1420 can obtain the training samples for the current round of training by calling the large model to execute the sample generation method based on the large model as described above. In another example, the processing module 1420 can obtain the large model obtained by the current round of training by calling the large model to execute the training method of the large model as described above. In another example, the processing module 1420 can also obtain the information processing result by calling the large model to execute the information processing method based on the large model as described above.

[0173] In a specific example, the processing module 1420 may include a control module 1421 , a storage module 1422 , and a calculation module 1423 .

[0174] The control module 1421 is the core support for the AI ​​agent 1400 to handle complex tasks. In the example, the control module 1421 will continuously interact with the storage module 1422, the operation module 1423 and / or the output module 1430 during operation. The performance of the control module 1421 can be closely related to the large model on which the AI ​​agent 1400 is based. In order to give full play to the capabilities of the large model, the internal structure of the control module 1421 can be designed to be highly configurable and scalable to cope with various types of tasks and requirements in real scenarios.

[0175] The storage module 1422 may be responsible for memorizing information such as historical conversations and event flows. The model information of the target big model as described above may be included in the storage module 1422. In the example, after the AI ​​agent 1400 obtains the information to be processed, the AI ​​agent 1400 may retrieve the model information big model of the target big model from the storage module 1422 and feed it back to the control module 1421. Then, the control module 1421 may process the big model of the information to be processed using the target big model, obtain the big model information processing result, and pass the big model of the information processing result to the output module 1430.

[0176] The operation module 1423 can be regarded as a predefined tool library. The quality assessment tool for performing quality assessment on the candidate answer information as described above can be included in the operation module 1423. In the example, when the AI ​​agent 1400 needs to perform quality assessment on the candidate answer information, the relevant quality assessment tool can be called from the operation module 1423 and fed back to the control module 1421. Then, the control module 1421 can use the quality assessment tool fed back to perform quality assessment on the candidate answer information, obtain the assessment value, and pass the assessment value to the output module 1430. It can be understood that although the large model has excellent language understanding and generation capabilities, it is like a human being, without the help of any tools, and the tasks that can be solved are very limited. When the AI ​​agent 1400 is given the ability to call tools, tasks such as completing quality assessment with the help of quality assessment tools can be achieved.

[0177] The output module 1450 may be used to output the output information obtained by the processing module 1420 .

[0178] In an example, when the processing module 1420 executes the sample generation method based on the large model as described above by calling the large model, the output information may be the training sample for the current round of training. In another example, when the processing module 1420 executes the training method of the large model as described above by calling the large model, the output information may be the large model obtained by the current round of training. In yet another example, when the processing module 1420 also executes the information processing method based on the large model as described above by calling the large model, the output information may be the information processing result.

[0179] The AI ​​agent 1400 according to the embodiment of the present disclosure can simply and effectively improve the level of intelligence, and improve flexibility and versatility.

[0180] Fig.15A block diagram of an electronic device suitable for implementing a sample generation method based on a large model, a training method for a large model, and an information processing method based on a large model according to an embodiment of the present disclosure is schematically shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0181] like Fig.15 As shown, the device 1500 includes a computing unit 1501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1502 or a computer program loaded from a storage unit 1508 into a random access memory (RAM) 1503. In the RAM 1503, various programs and data required for the operation of the device 1500 can also be stored. The computing unit 1501, the ROM 1502, and the RAM 1503 are connected to each other via a bus 1504. An input / output (I / O) interface 1505 is also connected to the bus 1504.

[0182] A number of components in the device 1500 are connected to the I / O interface 1505, including: an input unit 1506, such as a keyboard, a mouse, etc.; an output unit 1507, such as various types of displays, speakers, etc.; a storage unit 1508, such as a disk, an optical disk, etc.; and a communication unit 1509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1509 allows the device 1500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0183] The computing unit 1501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1501 performs the various methods and processes described above, such as a sample generation method based on a large model, a training method for a large model, and an information processing method based on a large model. For example, in some embodiments, a sample generation method based on a large model, a training method for a large model, and an information processing method based on a large model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1500 via the ROM 1502 and / or the communication unit 1509. When the computer program is loaded into RAM 1503 and executed by the computing unit 1501, one or more steps of the large model-based sample generation method, large model-based training method, and large model-based information processing method described above may be performed. Alternatively, in other embodiments, the computing unit 1501 may be configured to perform the large model-based sample generation method, large model-based training method, and large model-based information processing method in any other appropriate manner (e.g., by means of firmware).

[0184] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0185] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0186] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0187] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0188] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0189] A computer system may include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises through computer programs running on respective computers and having a client-server relationship to each other. The server may be a cloud server, a server in a distributed system, or a server combined with a blockchain.

[0190] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0191] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A sample generation method based on a large model, comprising: By calling the large model obtained through the previous round of training, according to the prompt information, the target reference information used for the current round of training is processed to obtain at least one question information and a plurality of candidate answer information for each of the question information; as well as For each piece of question information, a training sample for a current round of training is generated according to the question information and at least two target answer information among the plurality of candidate answer information.

2. The method according to claim 1, wherein: The target reference information includes a first number of initial question information and actual answer information of each of the initial question information; The method of calling the large model obtained through the previous round of training and processing the target reference information used for the current round of training according to the prompt information to obtain at least one question information and a plurality of candidate answer information for each of the question information includes: For each of the initial question information, in response to obtaining a question generation instruction in the prompt information, input the question generation instruction, the initial question information and the actual answer information into the large model to obtain the at least one question information; and For each of the question information, in response to obtaining an answer generation instruction in the prompt information, the answer generation instruction and the question information are input into the large model to obtain a plurality of candidate answer information for each of the question information.

3. The method according to claim 1 or 2, wherein: The step of generating a training sample for a current round of training according to the question information and at least two target answer information among the plurality of candidate answer information comprises: Based on a preset rule, determining the at least two target answer information from the plurality of candidate answer information; and The training samples for the current round of training are generated according to the question information and the at least two target answer information.

4. The method according to claim 3 further comprises, before determining the at least two target answer information from the plurality of candidate answer information based on a preset rule: In response to obtaining the quality evaluation instruction in the prompt information, the quality evaluation instruction and the candidate answer information are input into the large model to obtain the evaluation value of each of the plurality of candidate answer information.

5. The method according to claim 4, wherein: The preset rule is used to screen the plurality of candidate answer information based on the evaluation value; The determining, based on a preset rule, the at least two target answer information from the plurality of candidate answer information comprises: determining at least one candidate answer information corresponding to the highest evaluation value among the plurality of evaluation values ​​as the target answer information; as well as At least one candidate answer information corresponding to the lowest evaluation value among the plurality of evaluation values ​​is determined as the target answer information.

6. The method according to claim 4, wherein: Generating training samples for the current round of training according to the question information and the at least two target answer information includes: For each piece of question information, generating a sample pair of the question information according to the question information, the candidate answer information corresponding to the highest evaluation value, and the candidate answer information corresponding to the lowest evaluation value; and The training samples for the current round of training are generated according to the at least one question information and the sample pairs of each of the question information.

7. The method according to claim 1, wherein: The target reference information for the current round of training is obtained in the following manner: According to a preset ratio, determining a first number of initial question information and actual answer information of each of the initial question information in the initial reference information, and obtaining the target reference information for the current round of training; The initial reference information includes a second number of initial question information and actual answer information of each of the initial question information, the preset ratio is the ratio of the first number to the second number, and the second number is greater than the first number.

8. A large model training method, comprising: Inputting at least one question information in the training samples for the current round of training into the large model obtained through the previous round of training, respectively, to obtain output information of each of the question information, wherein the training samples for the current round of training are obtained by using the method described in claims 1 to 7, and the training samples for the current round of training also include at least two target answer information of each of the question information; and According to the output information of each of the question information and the at least two target answer information, the large model obtained by the previous round of training is trained to obtain the large model obtained by the current round of training.

9. The method according to claim 8, wherein: The at least two target answer information include answer information corresponding to the highest evaluation value and answer information corresponding to the lowest evaluation value; In the case where the current round is greater than 1, training the large model obtained by the previous round of training according to the output information of each of the question information and the at least two target answer information, and obtaining the large model obtained by the current round of training includes: For each of the question information, determine a target loss function value according to a first loss function value determined based on the output information and the answer information corresponding to the highest evaluation value, and a second loss function value determined based on the output information and the answer information corresponding to the lowest evaluation value; as well as Based on the target loss function value, the model parameters of the large model obtained by the previous round of training are adjusted to obtain the large model obtained by the current round of training.

10. The method according to claim 9, further comprising, after training the large model obtained by the previous round of training according to the output information of each of the question information and the at least two target answer information to obtain the large model obtained by the current round of training: Using the test sample to test the large model obtained through the current round of training to obtain a test result; and In response to the test result satisfying a predetermined condition, the large model obtained through the current round of training is determined as the target large model.

11. The method according to claim 8, wherein: When the current round is equal to 1, the method further includes: Acquire initial reference information, wherein the initial reference information includes a second number of initial question information and actual answer information of each of the initial question information; and The initial reference information is used to train the initial large model to obtain a large model obtained through the first round of training.

12. An information processing method based on a large model, comprising: Get the information to be processed; as well as The information to be processed is input into a target large model to obtain an information processing result, wherein the target large model is obtained by using the method described in claims 8 to 11.

13. The method according to claim 12, wherein: The information to be processed and the initial reference information used to train the target large model belong to the same business field.

14. A sample generation device based on a large model, comprising: A processing module, used to process the target reference information used for the current round of training by calling the large model obtained by the previous round of training according to the prompt information, to obtain at least one question information and a plurality of candidate answer information for each of the question information; as well as A generating module is used to generate, for each of the question information, a training sample for the current round of training according to the question information and at least two target answer information among the multiple candidate answer information.

15. The device according to claim 14, wherein: The target reference information includes a first number of initial question information and actual answer information of each of the initial question information; The processing module comprises: A first processing unit is configured to, for each of the initial question information, input the initial question information and the actual answer information of the question generation instruction into the large model in response to obtaining the question generation instruction in the prompt information, so as to obtain the at least one question information; as well as The second processing unit is used for, for each of the question information, in response to obtaining the answer generation instruction in the prompt information, inputting the answer generation instruction and the question information into the large model to obtain multiple candidate answer information for each of the question information.

16. The device according to claim 14 or 15, wherein: The generation module comprises: A first determining unit, configured to determine the at least two target answer information from the plurality of candidate answer information based on a preset rule; and A generating unit is used to generate the training samples for the current round of training according to the question information and the at least two target answer information.

17. The apparatus according to claim 16, further comprising: An evaluation unit is used to respond to the quality evaluation instruction obtained in the prompt information, input the quality evaluation instruction and the candidate answer information into the large model, and obtain the evaluation value of each of the multiple candidate answer information.

18. The device according to claim 17, wherein: The preset rule is used to screen the plurality of candidate answer information based on the evaluation value; The first determining unit includes: A first determining subunit is used to determine at least one candidate answer information corresponding to the highest evaluation value among the multiple evaluation values ​​as the target answer information; as well as The second determining subunit is configured to determine at least one candidate answer information corresponding to the lowest evaluation value among the plurality of evaluation values ​​as the target answer information.

19. The device according to claim 17, wherein: The generating unit comprises: A first generating subunit is configured to generate, for each of the question information, a sample pair of the question information according to the question information, the candidate answer information corresponding to the highest evaluation value, and the candidate answer information corresponding to the lowest evaluation value; and The second generating subunit is used to generate the training samples for the current round of training according to the at least one question information and the sample pairs of each question information.

20. The device according to claim 14, wherein: The target reference information for the current round of training is obtained in the following manner: According to a preset ratio, determining a first number of initial question information and actual answer information of each of the initial question information in the initial reference information, and obtaining the target reference information for the current round of training; The initial reference information includes a second number of initial question information and actual answer information of each of the initial question information, the preset ratio is the ratio of the first number to the second number, and the second number is greater than the first number.

21. A large model training device, comprising: A first input module, used to input at least one question information in the training samples for the current round of training into the large model obtained by the previous round of training, to obtain output information of each of the question information, wherein the training samples for the current round of training are obtained by using the apparatus of claims 14 to 20, and the training samples for the current round of training also include at least two target answer information of each of the question information; and The first training module is used to train the large model trained in the previous round according to the output information of each of the question information and the at least two target answer information, so as to obtain the large model obtained by the current round of training.

22. The device according to claim 21, wherein The at least two target answer information include answer information corresponding to the highest evaluation value and answer information corresponding to the lowest evaluation value; When the current round is greater than 1, the first training module includes: A second determining unit is used to determine, for each of the question information, a target loss function value according to a first loss function value determined based on the output information and the answer information corresponding to the highest evaluation value, and a second loss function value determined based on the output information and the answer information corresponding to the lowest evaluation value; as well as An adjustment unit is used to adjust the model parameters of the large model obtained by the previous round of training based on the target loss function value to obtain the large model obtained by the current round of training.

23. The apparatus according to claim 22, further comprising: A testing module, used to test the large model trained in the current round using a test sample to obtain a test result; as well as A determination module is used to determine the large model obtained through the current round of training as the target large model in response to the test result satisfying a predetermined condition.

24. The device according to claim 21, wherein When the current round is equal to 1, the device further includes: A first acquisition module is used to acquire initial reference information, wherein the initial reference information includes a second number of initial question information and actual answer information of each of the initial question information; and The second training module is used to train the initial large model using the initial reference information to obtain the large model obtained through the first round of training.

25. An information processing device based on a large model, comprising: A second acquisition module is used to acquire information to be processed; as well as The second input module is used to input the information to be processed into the target large model to obtain the information processing result, wherein the target large model is obtained using the device described in claims 21 to 24.

26. The device according to claim 25, wherein The information to be processed and the initial reference information used to train the target large model belong to the same business field.

27. An intelligent agent comprises: An input module, used for receiving input information; A processing module, used to determine a target task based on the input information received by the input module, determine a large model based on the target task, and obtain output information by calling the large model to execute the method of claim 12 or 13; as well as An output module is used to output the output information obtained by the processing module.

28. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, The method comprises the step of executing the one or more computer programs to implement the steps of the method according to any one of claims 1 to 13.

29. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

30. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images