Answer generation method and device, equipment and storage medium

Abstract

The invention belongs to the field of computers, and discloses an answer generation method and device, equipment and a storage medium. The method comprises the steps that when a model prediction instruction of a user is received, human information and user demand information corresponding to the model prediction instruction are acquired; determining a personalized model corresponding to the personal information; and responding to the user demand information through the personalized model, and generating answer information corresponding to the user demand information. The personalized model is determined according to the personal information in the model prediction instruction, the user demand information is responded through the personalized model, and the answer information corresponding to the user demand information is generated. Compared with an existing mode of answering the questions proposed by the user through a general large model, the mode provided by the invention can select the corresponding personalized model to respond to the user demand information according to the personal information of the user, so that the obtained answer information better conforms to the personal information of the user.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an answer generation method, device, equipment and storage medium. Background Art

[0002] Intelligent models such as ChatGPT fully demonstrate the capabilities of large language models in generation and reasoning, making the development and application of personal intelligent assistants possible. For example, the RLHF algorithm shows the effectiveness of reinforcement learning in aligning large language models. Delta tuning algorithms such as LoRA provide a way to fine-tune the parameters of the original large model without changing them. However, when users use these models for question and answer, they are provided with a unified dialogue service, without personalized services for users, and cannot meet their personalized needs.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide an answer generation method, device, equipment and storage medium, aiming to solve the technical problem that the existing large language model provides users with unified dialogue services and cannot meet the personalized needs of users.

[0005] To achieve the above object, the present invention provides an answer generation method, which comprises the following steps:

[0006] Upon receiving a model prediction instruction from a user, obtaining the persona information and user demand information corresponding to the model prediction instruction;

[0007] Determine a personalized model corresponding to the personality information;

[0008] The personalized model is used to respond to the user demand information and generate answer information corresponding to the user demand information.

[0009] Optionally, before the step of determining the personalized model corresponding to the personality information, the method further includes:

[0010] Acquire preset character information, wherein the preset character information includes each character and a character prompt word corresponding to the character;

[0011] The preset basic large language model is trained according to the preset personality information to obtain personalized models corresponding to each personality.

[0012] Optionally, after the step of training a preset basic large language model according to the preset persona information to obtain a personalized model corresponding to each persona, the step further includes:

[0013] Obtaining specific questions corresponding to each of the personas;

[0014] Inputting the specific question into the corresponding personalized model to obtain at least one answer information output by the personalized model;

[0015] The personalized model is subjected to intensive training according to the answer information and the specific question, and the personalized model after intensive training is used as the personalized model.

[0016] Optionally, the step of performing intensive training on the personalized model according to the answer information and the specific question includes:

[0017] Determine the answer score corresponding to each of the answer information;

[0018] Training an initial reward model according to the answer information and the answer score to obtain a target reward model;

[0019] The personalized model is reinforced trained based on the target reward model.

[0020] Optionally, the step of determining the answer score corresponding to each of the answer information includes:

[0021] Determine the answer quality score corresponding to the answer information according to the answer quality evaluation model;

[0022] Inputting the answer information and the specific question corresponding to the answer information into a preset personality conformity model to obtain a personality conformity score output by the preset personality conformity model;

[0023] The answer score corresponding to each of the answer information is determined according to the character setting conformity score and the answer quality score.

[0024] Optionally, the step of determining the answer score corresponding to each of the answer information according to the character setting conformity score and the answer quality score includes:

[0025] Get preset weight information;

[0026] Determine, according to the preset weight information, a first weight corresponding to the character setting conformity score and a second weight corresponding to the answer quality score;

[0027] Determine the answer score corresponding to each of the answer information based on the personality conformity score, the first weight, the answer quality score and the second weight.

[0028] Optionally, the step of performing intensive training on the personalized model based on the target reward model includes:

[0029] Based on the target reward model, the personalized model is reinforced trained by a delta tuning algorithm.

[0030] Optionally, the step of determining the answer score corresponding to each of the answer information includes:

[0031] The answer score corresponding to each of the answer information is determined by a pre-trained answer evaluation model.

[0032] Optionally, before the step of determining the answer score corresponding to each of the answer information by using the pre-trained answer evaluation model, the step further includes:

[0033] Answer information sample data is obtained, where the answer information sample data includes sample answer information and a sample answer score corresponding to the sample answer information.

[0034] The basic large language model is trained according to the answer information sample data to obtain an answer evaluation model.

[0035] Optionally, after the step of responding to the user demand information through the personalized model and generating answer information corresponding to the user demand information, the method further includes:

[0036] Obtaining evaluation information from the user based on the answer information feedback;

[0037] Generate target personality information according to the evaluation information and the personality information, wherein the target personality information includes a newly added target personality and a personality prompt word corresponding to the target personality;

[0038] Model training is performed on a preset basic large language model according to the target persona information to obtain a personalized model corresponding to the target persona.

[0039] In addition, to achieve the above object, the present invention also provides an answer generation device, the device comprising:

[0040] A receiving module, configured to obtain, upon receiving a model prediction instruction from a user, the persona information and user demand information corresponding to the model prediction instruction;

[0041] A determination module, used to determine a personalized model corresponding to the personality information;

[0042] The response module is used to respond to the user demand information through the personalized model and generate answer information corresponding to the user demand information.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also proposes an answer generation device, which includes: a memory, a processor, and an answer generation program stored in the memory and executable on the processor, wherein the answer generation program is configured to implement the steps of the answer generation method described above.

[0044] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which an answer generation program is stored, and when the answer generation program is executed by a processor, the steps of the answer generation method described above are implemented.

[0045] When the present invention receives a model prediction instruction from a user, it obtains the persona information and user demand information corresponding to the model prediction instruction; determines the personalized model corresponding to the persona information; responds to the user demand information through the personalized model, and generates answer information corresponding to the user demand information. Since the present invention determines the personalized model based on the persona information in the model prediction instruction, responds to the user demand information through the personalized model, and generates answer information corresponding to the user demand information. Compared with the existing method of answering questions raised by users through a general large model, the above method of the present invention can select the corresponding personalized model to respond to the user demand information based on the user's persona information, so that the answer information obtained is more in line with the user's persona. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the structure of an answer generation device in a hardware operating environment involved in an embodiment of the present invention;

[0047] Figure 2 It is a flowchart of the first embodiment of the answer generation method of the present invention;

[0048] Figure 3 It is a flowchart of a second embodiment of the answer generation method of the present invention;

[0049] Figure 4 A schematic diagram of a flow chart of a third embodiment of the answer generation method of the present invention;

[0050] Figure 5 It is a structural block diagram of the first embodiment of the answer generating device of the present invention.

[0051] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0053] Reference Figure 1 , Figure 1A schematic diagram of the structure of an answer generation device for the hardware operating environment involved in the embodiment of the present invention.

[0054] like Figure 1 As shown, the answer generation device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0055] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the answer generating device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0056] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an answer generation program.

[0057] exist Figure 1 In the answer generating device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the answer generating device of the present invention can be set in the answer generating device, and the answer generating device calls the answer generating program stored in the memory 1005 through the processor 1001, and executes the answer generating method provided by the embodiment of the present invention.

[0058] Based on the above answer generation device, the embodiment of the present invention provides an answer generation method, referring to Figure 2 , Figure 2 Schematic diagram of the flow chart of the first embodiment of the answer generation method of the present invention.

[0059] In this embodiment, the answer generation method includes the following steps:

[0060] Step S10: upon receiving a model prediction instruction from a user, obtaining the persona information and user demand information corresponding to the model prediction instruction.

[0061] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a mobile phone, a tablet computer, a personal computer, etc., or an electronic device or an answer generation device that can realize the above functions. The following takes the answer generation device as an example to illustrate this embodiment and the following embodiments.

[0062] It should be noted that the model prediction instruction may be a command triggered by a user to predict the needs input by the user through a personalized model. The model prediction instruction may include the user's personality information and user need information, and the user need information may be information such as questions input by the user, search keywords, etc.

[0063] Step S20: Determine the personalized model corresponding to the personality information.

[0064] It should be noted that the determination of the personalized model corresponding to the personality information may be to determine the personality pre-set by the user or the personality currently selected by the user according to the personality information. When the user does not select a personality, the default personality configured in advance by the user may be used, and the user may also select an instant personality when using it. The personalized model corresponding to the personality is determined according to the preset personality model mapping table. The preset personality model mapping table may be a mapping table for storing the correspondence between the personality and the personalized model.

[0065] Step S30: Respond to the user demand information through the personalized model to generate answer information corresponding to the user demand information.

[0066] It should be noted that responding to the user demand information through the personalized model and generating answer information corresponding to the user demand information may be generating answer information corresponding to the user demand information through the personalized model. Specifically, it may be: inputting the user demand information into the personalized model to obtain answer information corresponding to the user demand information generated by the personalized model.

[0067] When receiving a model prediction instruction from a user, this embodiment obtains the persona information and user demand information corresponding to the model prediction instruction; determines the personalized model corresponding to the persona information; responds to the user demand information through the personalized model, and generates answer information corresponding to the user demand information. Since this embodiment determines the personalized model based on the persona information in the model prediction instruction, responds to the user demand information through the personalized model, and generates answer information corresponding to the user demand information. Compared with the existing method of answering questions raised by users through a general large model, the above method of this embodiment can select the corresponding personalized model to respond to the user demand information based on the user's persona information, so that the answer information obtained is more in line with the user's persona.

[0068] refer to Figure 3 , Figure 3 Schematic diagram of the flow chart of the second embodiment of the answer generation method of the present invention.

[0069] Based on the first embodiment above, in this embodiment, before step S20, the following steps are further included:

[0070] Step S101: obtaining preset character information, wherein the preset character information includes each character and a character prompt word corresponding to the character.

[0071] It should be noted that the preset persona information may include several preset personas and persona prompts corresponding to the personas. For example, the preset personas may include: rigorous, humorous, etc. The persona prompts corresponding to the personas may be artificial intelligence (AI) prompts. Prompt is a method of using natural language to guide or inspire artificial intelligence models to complete specific tasks. For example, the persona prompt corresponding to the rigorous persona may be: You are a serious personal assistant. Each answer you give is as factual and logical as possible. The answer should be as concise as possible while meeting the requirements. If you need to explain the answer, ensure that the explanation is concise and logical. Do not provide additional information that is not relevant to the question, and do not make subjective judgments.

[0072] Step S102: Perform model training on a preset basic large language model according to the preset persona information to obtain a personalized model corresponding to each persona.

[0073] It should be noted that the preset basic large language model can be an ordinary general large language model. The model training of the preset basic large language model according to the preset personality information to obtain the personalized model corresponding to each personality can be inputting the personality prompt word and the question information into the general large language model, obtaining the answer information output by the general large language model, and performing SFT model training on the preset basic large language model based on the question information and the answer information to obtain the personalized model corresponding to each personality. Afterwards, when using the personalized model to generate the answer information corresponding to the personality, it is not necessary to input the personality prompt word, and the personalized model can generate the answer information corresponding to the personality, wherein the question information can be a specific question and / or a general question pre-written for the personality. For example, the preset personality information includes rigorous and humorous personality and the personality prompt words corresponding to these two personality, and the personality prompt words and question information corresponding to the rigorous personality are input into the general large language model to obtain the answer information output by the general large language model, and then the preset basic large language model is trained by the question information and the answer information to obtain the personalized model corresponding to the rigorous personality. The character prompt words and question information corresponding to the humorous character are input into the general large language model to obtain the answer information output by the general large language model, and then the preset basic large language model is trained with the question information and the answer information to obtain the personalized model corresponding to the humorous character. Through the above steps, the personalized model corresponding to each character is obtained.

[0074] Furthermore, in order to better meet the user's personalized needs for the large language model, after step S102, the method further includes: obtaining specific questions corresponding to each of the personas;

[0075] Inputting the specific question into the corresponding personalized model to obtain at least one answer information output by the personalized model;

[0076] The personalized model is subjected to intensive training according to the answer information and the specific question, and the personalized model after intensive training is used as the personalized model.

[0077] It should be noted that the specific questions corresponding to the persona may include specific questions pre-written for the persona and / or general questions used for model training. The inputting of the specific question into the corresponding personalized model to obtain at least one answer information output by the personalized model may be inputting the specific question corresponding to the persona into the personalized model corresponding to the persona to obtain at least one answer information corresponding to the specific question predicted by the personalized model. The intensive training of the personalized model according to the answer information and the specific question and using the personalized model after intensive training as the personalized model may be conducting a quality review of the answer information output by the personalized model, determining the correlation between the answer information and the specific question and the persona, and intensive training the personalized model according to the correlation between each answer information and the corresponding answer information to obtain a personalized model after intensive training.

[0078] Furthermore, in order to accurately evaluate the relevance between the answer information and the persona and obtain a personalized model that is more consistent with the persona, the step of intensively training the personalized model according to the answer information and the specific question includes:

[0079] Determine the answer score corresponding to each of the answer information;

[0080] Training an initial reward model according to the answer information and the answer score to obtain a target reward model;

[0081] The personalized model is reinforced trained based on the target reward model.

[0082] It should be noted that the answer scores corresponding to the answer information may be determined by manually marking the answer scores corresponding to the answer information. Alternatively, the answer scores corresponding to the answer information may be determined by a pre-trained answer evaluation model. The answer evaluation model may be a model pre-trained with sample data for predicting the answer scores corresponding to the answer information. Specifically, answer information sample data may be obtained, wherein the answer information sample data includes sample answer information and sample answer scores corresponding to the sample answer information. The basic large language model is trained according to the answer information sample data to obtain an answer evaluation model.

[0083] In specific implementation, the answer score corresponding to the answer information can be determined by manual annotation, and then based on the accumulated sample data, an answer evaluation model that specifically predicts the answer score can be trained to assist in manual annotation. The answer score corresponding to the answer information can be determined by model prediction + manual review. Finally, when the accumulated sample data is sufficient, the answer evaluation model can basically replace manual annotation.

[0084] It should be noted that the initial reward model may be an RM reward model. The training of the initial reward model according to the answer information and the answer score to obtain the target reward model may be a training reward model according to the answer information and the answer score corresponding to the answer information. The intensive training of the personalized model based on the target reward model may be intensive training of the personalized model based on the target reward model through a delta tuning algorithm. The delta tuning algorithm includes algorithms such as lora and qlora. In this embodiment, the personalized model is trained using a delta tuning algorithm, which enables each personalized model to share the capabilities of a basic large model (i.e., a general large language model) and reduce training costs. The specific steps of intensive training the personalized model based on the target reward model through a delta tuning algorithm may include: inputting a specific question corresponding to the person setting into the personalized model, obtaining the answer information generated by the personalized model, evaluating the answer information according to the target reward model, obtaining an evaluation result, and then the personalized model performs weight adjustment and optimization of the model based on the evaluation result to generate a better answer.

[0085] Furthermore, this embodiment can evaluate the answer information from two aspects: answer quality and answer-to-character consistency. Only when both aspects meet the requirements can the highest score be obtained. The above-mentioned training reward model includes evaluation of both answer quality and character consistency. However, for the accuracy of answer evaluation, the two can also be separated. The step of determining the answer score corresponding to each of the answer information includes:

[0086] Determine the answer quality score corresponding to the answer information according to the answer quality evaluation model;

[0087] Inputting the answer information and the specific question corresponding to the answer information into a preset personality conformity model to obtain a personality conformity score output by the preset personality conformity model;

[0088] The answer score corresponding to each of the answer information is determined according to the character setting conformity score and the answer quality score.

[0089] It should be noted that the answer quality evaluation model may be a pre-trained model for evaluating the quality of answer information. The preset persona conformity model may be a pre-trained model for evaluating the conformity between answer information and persona.

[0090] The inputting of the answer information and the specific question corresponding to the answer information into a preset personality conformity model to obtain the personality conformity score output by the preset personality conformity model may be to specify the current personality so that the preset personality conformity model outputs the personality conformity score corresponding to the answer information based on the specific question and the current personality. The determining of the answer score corresponding to each of the answer information according to the personality conformity score and the answer quality score may be to use the sum or average score of the personality conformity score and the answer quality score as the answer score corresponding to the answer information.

[0091] Furthermore, in order to accurately determine the answer score corresponding to the answer information, the step of determining the answer score corresponding to each of the answer information according to the personality conformity score and the answer quality score includes:

[0092] Get preset weight information;

[0093] Determine, according to the preset weight information, a first weight corresponding to the character setting conformity score and a second weight corresponding to the answer quality score;

[0094] Determine the answer score corresponding to each of the answer information based on the character setting conformity score, the first weight, the answer quality score and the second weight.

[0095] It should be noted that the preset weight information may include the weights of the influence of the character conformity score and the answer quality score on the answer score. The first weight may be the weight corresponding to the character conformity score. The second weight may be the weight corresponding to the answer quality score. The answer score corresponding to each answer information is determined based on the character conformity score, the first weight, the answer quality score and the second weight, and the answer score corresponding to the answer information is obtained by multiplying the character conformity score by the first weight plus the answer quality score by the second weight.

[0096] Furthermore, this embodiment can also iteratively train the reward model and the personalized model, specifically, it can be: obtaining the answer information output by the personalized model, determining the question information and answer evaluation corresponding to the answer information, training the reward model according to the answer information, answer evaluation and question information, obtaining the trained reward model, and then further strengthening the personalized model through the trained reward model. The reward model can include any one of the target reward model, the preset personality conformity model, the answer quality evaluation model and the answer evaluation model mentioned in the above embodiments.

[0097] This embodiment obtains preset person information, which includes each person and the person prompt words corresponding to the person; the preset basic large language model is trained according to the preset person information to obtain the personalized model corresponding to each person. This embodiment trains the preset basic large language model according to the person and the person prompt words corresponding to the person to obtain the personalized model corresponding to each person. When responding to user needs, it is possible to respond according to the personalized model corresponding to the person information, meet the user's personalized needs for the large language model, and make the answer information generated by the response more in line with the user's expectations.

[0098] refer to Figure 4 , Figure 4 Schematic diagram of the flow chart of the third embodiment of the answer generation method of the present invention.

[0099] Based on the above embodiments, in this embodiment, after step S30, the method further includes:

[0100] Step S40: Obtaining evaluation information provided by the user based on the answer information feedback.

[0101] It should be noted that the evaluation information may include new personality information input by the user, optimization information fed back for existing personality information, or prompt words for optimizing answer information.

[0102] Step S50: generating target personality information according to the evaluation information and the personality information, wherein the target personality information includes a newly added target personality and a personality prompt word corresponding to the target personality.

[0103] It should be noted that generating the target personality information based on the evaluation information and the personality information may be generating a target personality prompt word corresponding to the personality information based on the evaluation information. When the evaluation information includes a personality newly added by the user, the target personality information may also include a target personality prompt word corresponding to the newly added personality.

[0104] Step S60: Perform model training on a preset basic large language model according to the target persona information to obtain a personalized model corresponding to the target persona.

[0105] It should be noted that the model training of the preset basic large language model according to the target persona information to obtain the personalized model corresponding to the target persona may be that when the target persona information does not contain new persona information, the personalized model corresponding to the current persona in the target persona information is obtained, and the personalized model is optimized and trained according to the target persona prompt words in the target persona information.

[0106] When the target personal information includes new personal information, the newly added personal information and the corresponding target personal prompt words in the target personal information are obtained, and the preset basic large language model is model-trained according to the personal information and the target personal prompt words to obtain the personalized model corresponding to the newly added personal information. The model training of the preset basic large language model to obtain the personalized model corresponding to the newly added personal information can specifically refer to the steps of training the preset basic large language model according to the preset personal information to obtain the personalized models corresponding to each of the personalities in the above embodiment, which will not be repeated in this embodiment.

[0107] This embodiment obtains the evaluation information fed back by the user based on the answer information; generates target personality information based on the evaluation information and the personality information, wherein the target personality information includes a newly added target personality and a personality prompt word corresponding to the target personality; performs model training on a preset basic large language model based on the target personality information to obtain a personalized model corresponding to the target personality. This embodiment can receive evaluation information fed back by the user, and optimize the personalized model based on the evaluation information or train a personalized model corresponding to new personality information, which can meet the user's personalized needs for the model. Improve the accuracy of the answer information and user personality predicted by the model.

[0108] Reference Figure 5 , Figure 5 It is a structural block diagram of the first embodiment of the answer generating device of the present invention.

[0109] like Figure 5 As shown, the answer generation device proposed in the embodiment of the present invention includes:

[0110] The receiving module 10 is used to obtain the persona information and user demand information corresponding to the model prediction instruction when receiving the model prediction instruction from the user;

[0111] A determination module 20, configured to determine a personalized model corresponding to the personality information;

[0112] The response module 30 is used to respond to the user demand information through the personalized model and generate answer information corresponding to the user demand information.

[0113] When receiving a model prediction instruction from a user, this embodiment obtains the persona information and user demand information corresponding to the model prediction instruction; determines the personalized model corresponding to the persona information; responds to the user demand information through the personalized model, and generates answer information corresponding to the user demand information. Since this embodiment determines the personalized model based on the persona information in the model prediction instruction, responds to the user demand information through the personalized model, and generates answer information corresponding to the user demand information. Compared with the existing method of answering questions raised by users through a general large model, the above method of this embodiment can select the corresponding personalized model to respond to the user demand information based on the user's persona information, so that the answer information obtained is more in line with the user's persona.

[0114] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0115] In addition, for technical details not fully described in this embodiment, reference can be made to the answer generation method provided in any embodiment of the present invention, and will not be repeated here.

[0116] Based on the above-mentioned first embodiment of the answer generation device of the present invention, a second embodiment of the answer generation device of the present invention is proposed.

[0117] In this embodiment, the answer generation device further includes a model training module, the model training module is used to obtain preset character information, the preset character information includes each character and the character prompt word corresponding to the character;

[0118] The preset basic large language model is trained according to the preset personality information to obtain personalized models corresponding to each personality.

[0119] Furthermore, the model training module is also used to obtain specific questions corresponding to each of the personalities;

[0120] Inputting the specific question into the corresponding personalized model to obtain at least one answer information output by the personalized model;

[0121] The personalized model is subjected to intensive training according to the answer information and the specific question, and the personalized model after intensive training is used as the personalized model.

[0122] Furthermore, the model training module is also used to determine the answer score corresponding to each of the answer information;

[0123] Training an initial reward model according to the answer information and the answer score to obtain a target reward model;

[0124] The personalized model is reinforced trained based on the target reward model.

[0125] Furthermore, the model training module is also used to determine the answer quality score corresponding to the answer information according to the answer quality evaluation model;

[0126] Inputting the answer information and the specific question corresponding to the answer information into a preset personality conformity model to obtain a personality conformity score output by the preset personality conformity model;

[0127] The answer score corresponding to each of the answer information is determined according to the character setting conformity score and the answer quality score.

[0128] Furthermore, the model training module is also used to obtain preset weight information;

[0129] Determine, according to the preset weight information, a first weight corresponding to the character setting conformity score and a second weight corresponding to the answer quality score;

[0130] Determine the answer score corresponding to each of the answer information based on the character setting conformity score, the first weight, the answer quality score and the second weight.

[0131] Furthermore, the model training module is also used to perform intensive training on the personalized model based on the target reward model through a delta tuning algorithm.

[0132] Furthermore, the model training module is also used to determine the answer score corresponding to each of the answer information through a pre-trained answer evaluation model.

[0133] Furthermore, the model training module is also used to obtain answer information sample data, and the answer information sample data includes sample answer information and sample answer scores corresponding to the sample answer information.

[0134] The basic large language model is trained according to the answer information sample data to obtain an answer evaluation model.

[0135] Furthermore, the response module 30 is also used to obtain evaluation information of the user based on the answer information feedback;

[0136] Generate target personality information according to the evaluation information and the personality information, wherein the target personality information includes a newly added target personality and a personality prompt word corresponding to the target personality;

[0137] Model training is performed on a preset basic large language model according to the target persona information to obtain a personalized model corresponding to the target persona.

[0138] Other embodiments or specific implementations of the answer generation device of the present invention can refer to the above-mentioned method embodiments, which will not be described in detail here.

[0139] In addition, an embodiment of the present invention further proposes a storage medium, on which an answer generation program is stored. When the answer generation program is executed by a processor, the steps of the answer generation method described above are implemented.

[0140] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0141] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0143] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

[0144] The present invention discloses A1, a method for generating an answer, the method for generating an answer comprising the following steps:

[0145] Upon receiving a model prediction instruction from a user, obtaining the persona information and user demand information corresponding to the model prediction instruction;

[0146] Determine a personalized model corresponding to the personality information;

[0147] The personalized model is used to respond to the user demand information and generate answer information corresponding to the user demand information.

[0148] A2. The answer generation method as described in A1, before the step of determining the personalized model corresponding to the personality information, further comprising:

[0149] Acquire preset character information, wherein the preset character information includes each character and a character prompt word corresponding to the character;

[0150] The preset basic large language model is trained according to the preset personality information to obtain personalized models corresponding to each personality.

[0151] A3. The answer generation method as described in A2, after the step of training the preset basic large language model according to the preset persona information to obtain the personalized model corresponding to each persona, further includes:

[0152] Obtaining specific questions corresponding to each of the personas;

[0153] Inputting the specific question into the corresponding personalized model to obtain at least one answer information output by the personalized model;

[0154] The personalized model is subjected to intensive training according to the answer information and the specific question, and the personalized model after intensive training is used as the personalized model.

[0155] A4. In the answer generation method described in A3, the step of performing intensive training on the personalized model according to the answer information and the specific question comprises:

[0156] Determine the answer score corresponding to each of the answer information;

[0157] Training an initial reward model according to the answer information and the answer score to obtain a target reward model;

[0158] The personalized model is reinforced trained based on the target reward model.

[0159] A5. In the answer generation method as described in A4, the step of determining the answer score corresponding to each of the answer information comprises:

[0160] Determine the answer quality score corresponding to the answer information according to the answer quality evaluation model;

[0161] Inputting the answer information and the specific question corresponding to the answer information into a preset personality conformity model to obtain a personality conformity score output by the preset personality conformity model;

[0162] The answer score corresponding to each of the answer information is determined according to the character setting conformity score and the answer quality score.

[0163] A6. The answer generation method as described in A5, wherein the step of determining the answer score corresponding to each of the answer information according to the character setting conformity score and the answer quality score comprises:

[0164] Get preset weight information;

[0165] Determine, according to the preset weight information, a first weight corresponding to the character setting conformity score and a second weight corresponding to the answer quality score;

[0166] Determine the answer score corresponding to each of the answer information based on the personality conformity score, the first weight, the answer quality score and the second weight.

[0167] A7. In the answer generation method described in A4, the step of performing intensive training on the personalized model based on the target reward model comprises:

[0168] Based on the target reward model, the personalized model is reinforced trained by a delta tuning algorithm.

[0169] A8. In the answer generation method as described in A4, the step of determining the answer score corresponding to each of the answer information comprises:

[0170] The answer score corresponding to each of the answer information is determined by a pre-trained answer evaluation model.

[0171] A9. The answer generation method as described in A8, before the step of determining the answer score corresponding to each of the answer information by using the pre-trained answer evaluation model, further includes:

[0172] Answer information sample data is obtained, where the answer information sample data includes sample answer information and a sample answer score corresponding to the sample answer information.

[0173] The basic large language model is trained according to the answer information sample data to obtain an answer evaluation model.

[0174] A10. The answer generation method as described in any one of A1 to A9, after the step of responding to the user demand information by the personalized model and generating answer information corresponding to the user demand information, further comprising:

[0175] Obtaining evaluation information from the user based on the answer information feedback;

[0176] Generate target personality information according to the evaluation information and the personality information, wherein the target personality information includes a newly added target personality and a personality prompt word corresponding to the target personality;

[0177] Model training is performed on a preset basic large language model according to the target persona information to obtain a personalized model corresponding to the target persona.

[0178] The present invention discloses B11, an answer generating device, the answer generating device comprising:

[0179] A receiving module, configured to obtain, upon receiving a model prediction instruction from a user, the persona information and user demand information corresponding to the model prediction instruction;

[0180] A determination module, used to determine a personalized model corresponding to the personality information;

[0181] The response module is used to respond to the user demand information through the personalized model and generate answer information corresponding to the user demand information.

[0182] B12. The answer generation device as described in B11, wherein the answer generation device further comprises a model training module, wherein the model training module is used to obtain preset character information, wherein the preset character information comprises each character and a character prompt word corresponding to the character;

[0183] The preset basic large language model is trained according to the preset personality information to obtain personalized models corresponding to each personality.

[0184] B13, the answer generation device as described in B12, wherein the model training module is further used to obtain specific questions corresponding to each of the personalities;

[0185] Inputting the specific question into the corresponding personalized model to obtain at least one answer information output by the personalized model;

[0186] The personalized model is subjected to intensive training according to the answer information and the specific question, and the personalized model after intensive training is used as the personalized model.

[0187] B14, the answer generation device as described in B13, wherein the model training module is further used to determine the answer score corresponding to each of the answer information;

[0188] Training an initial reward model according to the answer information and the answer score to obtain a target reward model;

[0189] The personalized model is reinforced trained based on the target reward model.

[0190] B15, the answer generation device as described in B14, the model training module is further used to determine the answer quality score corresponding to the answer information according to the answer quality evaluation model;

[0191] Inputting the answer information and the specific question corresponding to the answer information into a preset personality conformity model to obtain a personality conformity score output by the preset personality conformity model;

[0192] The answer score corresponding to each of the answer information is determined according to the character setting conformity score and the answer quality score.

[0193] B16, the answer generation device as described in B15, wherein the model training module is further used to obtain preset weight information;

[0194] Determine, according to the preset weight information, a first weight corresponding to the character setting conformity score and a second weight corresponding to the answer quality score;

[0195] Determine the answer score corresponding to each of the answer information based on the personality conformity score, the first weight, the answer quality score and the second weight.

[0196] B17. In the answer generation device as described in B14, the model training module is further used to perform intensive training on the personalized model through a delta tuning algorithm based on the target reward model.

[0197] B18. In the answer generation device as described in B14, the model training module is further used to determine the answer score corresponding to each of the answer information through a pre-trained answer evaluation model.

[0198] The present invention discloses C19, an answer generating device, which comprises: a memory, a processor and an answer generating program stored in the memory and executable on the processor, wherein the answer generating program is configured to implement the steps of the answer generating method described in any one of A1 to A10.

[0199] The present invention discloses D20, a storage medium, on which an answer generation program is stored. When the answer generation program is executed by a processor, the steps of the answer generation method as described in any one of A1 to A10 are implemented.

Claims

1. A method for generating an answer, characterized in that: The answer generation method comprises the following steps: Upon receiving a model prediction instruction from a user, obtaining the persona information and user demand information corresponding to the model prediction instruction; Determine a personalized model corresponding to the personality information; The personalized model is used to respond to the user demand information and generate answer information corresponding to the user demand information.

2. The answer generation method according to claim 1, characterized in that: Before the step of determining the personalized model corresponding to the personality information, the method further includes: Acquire preset character information, wherein the preset character information includes each character and a character prompt word corresponding to the character; The preset basic large language model is trained according to the preset personality information to obtain personalized models corresponding to each personality.

3. The answer generation method according to claim 2, characterized in that: After the step of training the preset basic large language model according to the preset persona information to obtain the personalized model corresponding to each persona, the method further includes: Obtaining specific questions corresponding to each of the personas; Inputting the specific question into the corresponding personalized model to obtain at least one answer information output by the personalized model; The personalized model is subjected to intensive training according to the answer information and the specific question, and the personalized model after intensive training is used as the personalized model.

4. The answer generation method according to claim 3, characterized in that: The step of performing intensive training on the personalized model according to the answer information and the specific question comprises: Determine the answer score corresponding to each of the answer information; Training an initial reward model according to the answer information and the answer score to obtain a target reward model; The personalized model is reinforced trained based on the target reward model.

5. The answer generation method according to claim 4, characterized in that: The step of determining the answer score corresponding to each of the answer information includes: Determine the answer quality score corresponding to the answer information according to the answer quality evaluation model; Inputting the answer information and the specific question corresponding to the answer information into a preset personality conformity model to obtain a personality conformity score output by the preset personality conformity model; The answer score corresponding to each of the answer information is determined according to the character setting conformity score and the answer quality score.

6. The answer generation method according to claim 5, characterized in that: The step of determining the answer score corresponding to each of the answer information according to the character setting conformity score and the answer quality score comprises: Get preset weight information; Determine, according to the preset weight information, a first weight corresponding to the character setting conformity score and a second weight corresponding to the answer quality score; Determine the answer score corresponding to each of the answer information based on the personality conformity score, the first weight, the answer quality score and the second weight.

7. The answer generation method according to claim 4, characterized in that: The step of performing intensive training on the personalized model based on the target reward model comprises: Based on the target reward model, the personalized model is reinforced trained by a delta tuning algorithm.

8. An answer generation device, characterized in that: The answer generating device comprises: A receiving module, configured to obtain, upon receiving a model prediction instruction from a user, the persona information and user demand information corresponding to the model prediction instruction; A determination module, used to determine a personalized model corresponding to the personality information; The response module is used to respond to the user demand information through the personalized model and generate answer information corresponding to the user demand information.

9. An answer generating device, characterized in that: The device comprises: a memory, a processor, and an answer generation program stored in the memory and executable on the processor, wherein the answer generation program is configured to implement the steps of the answer generation method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores an answer generation program, which, when executed by a processor, implements the steps of the answer generation method according to any one of claims 1 to 7.

Images