A method and device for generating a scenario question-answering model

Abstract

The present invention provides a method and device for generating a scenario question-answering model, and relates to the field of artificial intelligence technology. The method comprises: obtaining a plurality of sample question-answering pairs corresponding to each preset personality type; for each of the preset personality types, using the plurality of sample question-answering pairs to train a basic model to obtain a personality model corresponding to the preset personality type; using the personality model to generate a scenario answer corresponding to the scenario question, and obtaining a preference score of the personality model based on the scenario answer; based on the preference score of each personality model, fusing all personality models into a target question-answering model in a specified scenario. When answering questions in a specified scenario, the target question-answering model obtained by the fusion of the present invention can give answers that are more in line with user preferences, and can effectively improve the naturalness of the conversation of the question-answering model in the specified scenario.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for generating a scenario question-answering model. Background Art

[0002] With the rapid development of artificial intelligence technology, intelligent question answering has become a hot topic in the field of natural language processing. Intelligent question answering aims to provide users with a more natural and user-friendly question answering experience by simulating the way humans communicate.

[0003] To achieve this goal, preference alignment is usually used to make the model output consistent with human expectations and preferences. Human preference alignment refers to making the capabilities and behaviors of large language models consistent with human values, true intentions, and ethical principles, ensuring safety and trust in the process of human-AI collaboration. However, current preference alignment is only applicable to general language models or general tasks. In specific scenarios, due to the lack of understanding of industry-specific knowledge and the ability to adapt to user personalized needs, the output of the model is difficult to meet user expectations and preferences, resulting in poor naturalness and quality of conversations in specific scenarios. Summary of the invention

[0004] In response to the above problems, the purpose of the present invention is to provide a scenario question-answering model generation method and device, which can align the output of the question-answering model with human expectations and preferences in a specified scenario, so as to improve the naturalness of the conversation of the question-answering model in a specific scenario.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In one aspect, the present invention provides a method for generating a scenario question answering model, comprising:

[0007] Acquire multiple sample question-answer pairs corresponding to each preset personality type, wherein the sample question-answer pairs include a sample question, a first sample answer, and a second sample answer, wherein the first sample answer matches the preset personality type, and the second sample answer does not match the preset personality type;

[0008] For each of the preset personality types, use the multiple sample question-answer pairs to train a basic model to obtain a personality model corresponding to the preset personality type, wherein an output distribution of the personality model is aligned with the first sample answer;

[0009] Generate a scenario answer corresponding to a scenario question using the personality model, and obtain a preference score of the personality model based on the scenario answer, wherein the preference score is positively correlated with human preference, and the scenario question is a question under a specified scenario;

[0010] Based on the preference score of each personality model, all personality models are merged into a target question-answering model in the specified scenario.

[0011] Optionally, obtaining a plurality of sample question-answer pairs corresponding to each preset personality type includes:

[0012] Acquire a basic dialogue data set, wherein the basic dialogue data set includes a plurality of sample questions;

[0013] For each of the sample questions, a plurality of sample answers corresponding to the sample question are obtained through the basic model, wherein one sample answer corresponds to one preset personality type;

[0014] For each of the sample responses, calculating an output value corresponding to the sample response;

[0015] Calculate the output difference of each sample pair with the output value corresponding to each sample answer, wherein the sample pair includes a first sample answer and a second sample answer that meet a preset condition, and the output difference is the difference between the output value of the first sample answer and the output value of the second sample answer;

[0016] If the output difference is greater than a preset threshold, the first sample answer, the second sample answer, and the sample question are combined as a sample question-answer pair of a preset personality type corresponding to the first sample answer and the second sample answer.

[0017] Optionally, for each of the sample replies, calculating an output value corresponding to the sample reply includes:

[0018] For each sample answer, obtain the original score matrix generated when the base model predicts the sample answer;

[0019] For each row in the raw score matrix, obtain the column index corresponding to the sample answer and the maximum raw score in the row;

[0020] Calculate the difference between the score corresponding to the column index in the raw score matrix and the maximum raw score;

[0021] Based on the difference, an output value corresponding to the sample response is calculated.

[0022] Optionally, if the output difference is greater than a preset threshold, combining the first sample answer, the second sample answer, and the sample question as a sample question-answer pair of a preset personality type corresponding to the first sample answer and the second sample answer includes:

[0023] If the output difference is greater than a preset threshold, combining the sample question, the first sample answer, and the second sample answer in sequence as a sample question-answer pair of a preset personality type corresponding to the first sample answer;

[0024] The sample question, the second sample answer and the first sample answer are combined in sequence as a sample question-answer pair of a preset personality type corresponding to the second sample answer.

[0025] Optionally, for each of the sample questions, obtaining a plurality of sample answers corresponding to the sample question through a basic model includes:

[0026] For each of the sample questions, generating a personality template corresponding to each preset personality type based on a designated prompt word template, wherein the personality template includes a personality description corresponding to the preset personality type;

[0027] Combining the sample question with each of the personality templates to obtain a plurality of sample inputs corresponding to the sample question;

[0028] For each of the sample inputs, the sample input is inferred through the basic model to obtain a sample response that matches each of the preset personality types.

[0029] Optionally, for each of the preset personality types, using the multiple sample question-answer pairs to train a basic model to obtain a personality model corresponding to the preset personality type includes:

[0030] For each of the preset personality types, the basic model is copied to obtain a reference model;

[0031] Inputting the sample question into the basic model to obtain a first predicted probability of generating the first sample answer and a second predicted probability of generating the second sample answer;

[0032] Inputting the sample question into the reference model to obtain a first reference probability of generating the first sample answer and a second reference probability of generating the second sample answer;

[0033] constructing a loss function based on the first predicted probability, the second predicted probability, the first reference probability, and the second reference probability;

[0034] If the loss value of the loss function does not meet the preset convergence condition, adjusting the model parameters of the basic model;

[0035] If the loss value of the loss function satisfies the preset convergence condition, a personality model corresponding to the preset personality type is obtained.

[0036] Optionally, the using the personality model to generate a scenario answer corresponding to the scenario question, and obtaining a preference score of the personality model based on the scenario answer, includes:

[0037] Obtain a scenario problem set under a specified scenario, wherein the scenario problem set includes multiple scenario problems;

[0038] randomly extracting two personality models from the plurality of personality models as a first personality model and a second personality model, and randomly extracting a designated question from the scenario question set;

[0039] inputting the designated question into the first personality model and the second personality model respectively, obtaining a first designated answer corresponding to the first personality model and a second designated answer corresponding to the second personality model;

[0040] Based on the comparison result of the first designated answer and the second designated answer, the preference scores of the first personality model and the second personality model are updated until the number of comparisons is greater than a preset number, and the preference score of each personality model is obtained.

[0041] Optionally, the step of fusing all personality models into a target question-answering model in the specified scenario based on the preference score of each personality model includes:

[0042] For each personality model, using the difference between the preference scores of other personality models and the preference score of the personality model, calculate the score to be used of the personality model;

[0043] Normalizing the to-be-used scores corresponding to all the personality models to obtain a fusion weight corresponding to each personality model;

[0044] According to the fusion weight corresponding to each personality model, all personality models are fused into the target question-answering model for the specified scenario.

[0045] Optionally, the step of fusing all personality models into a target question-answering model in a specified scenario according to the fusion weight corresponding to each personality model includes:

[0046] Obtaining a model parameter vector corresponding to each personality model;

[0047] Determining a first model to be fused and a second model to be fused from the multiple personality models;

[0048] Using the fusion weight of the first model to be fused and the fusion weight of the second model to be fused, the model parameter vector of the first model to be fused and the model parameter vector of the second model to be fused are fused to obtain an intermediate fusion vector;

[0049] Determining the model parameter vector of the first model to be fused by using the intermediate fusion vector, and determining a new second model to be fused from the remaining personality models;

[0050] Returning to the step of fusing the model parameter vector of the first model to be fused and the model parameter vector of the second model to be fused using the fusion weight of the first model to be fused and the fusion weight of the second model to be fused to obtain an intermediate fusion vector, until all the personality models are traversed;

[0051] If all the personality models are traversed, the target question-answering model in the specified scenario is obtained based on the intermediate fusion vector.

[0052] On the other hand, the present invention also provides a scenario question-answering model generation device, which is used to implement any of the above methods, including:

[0053] an acquisition module, configured to acquire a plurality of sample question-answer pairs corresponding to each preset personality type, wherein the sample question-answer pairs include a sample question, a first sample answer, and a second sample answer, wherein the first sample answer matches the preset personality type, and the second sample answer does not match the preset personality type;

[0054] A training module, for training a basic model using the plurality of sample question-answer pairs for each of the preset personality types, to obtain a personality model corresponding to the preset personality type, wherein an output distribution of the personality model is aligned with the first sample answer;

[0055] A scenario processing module, used to generate a scenario answer corresponding to a scenario question using the personality model, and obtain a preference score of the personality model based on the scenario answer, wherein the preference score is positively correlated with human preference, and the scenario question is a question under a specified scenario;

[0056] A fusion module is used to fuse all personality models into a target question-answering model in the specified scenario based on the preference score of each personality model.

[0057] On the other hand, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores multiple instructions; the processor loads instructions from the memory to execute the steps in any one of the scenario question and answer model generation methods provided by the present invention.

[0058] On the other hand, the present invention also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in any scenario question and answer model generation method provided by the present invention.

[0059] On the other hand, the present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps in any one of the scenario question and answer model generation methods provided by the present invention.

[0060] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0061] In the embodiment of the present invention, a plurality of sample question-answer pairs corresponding to each preset personality type are used to train a basic model to obtain a personality model corresponding to the preset personality type, so that the output of each personality model matches the preset personality type. The preference score of the personality model is then calculated using the personality model's response in a specified scenario to measure whether the personality model's response in the specified scenario meets human preferences. Finally, all personality models are fused according to the preference score to obtain a target question-answer model, so that the target question-answer model can give answers that are more in line with user preferences when answering questions in the specified scenario, which can effectively improve the naturalness of the conversation of the question-answer model in the specified scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 It is a schematic diagram of an application scenario of the scenario question-answering model generation method provided by an embodiment of the present invention;

[0064] Figure 2 It is a flowchart of a method for generating a scenario question-answering model provided by an embodiment of the present invention;

[0065] Figure 3 is a schematic diagram of a process for obtaining preference scores of various personality models provided by an embodiment of the present invention;

[0066] Figure 4 It is a structural diagram of a scenario question-answering model generating device provided by an embodiment of the present invention;

[0067] Figure 5 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0069] It is understandable that in the specific implementation of the present invention, data related to user information, etc., needs to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0070] The embodiment of the present invention proposes a scenario question-answering model generation method to improve the naturalness of conversation in a specific scenario. Figure 1 , shows a schematic diagram of an application scenario of the scenario question answering model generation method. The application scenario may include a server 101 and a database 102, and data can be exchanged between the server 101 and the database 102 via a network. The server 101 may be a single server or a server cluster consisting of multiple servers.

[0071] Database 102 may store various data sets, such as multiple sample question-answer pairs corresponding to each preset personality type. Server 101 may obtain multiple sample question-answer pairs corresponding to each preset personality type from database 102, wherein the sample question-answer pairs include sample questions, first sample replies, and second sample replies, wherein the first sample replies match the preset personality type, and the second sample replies do not match the preset personality type; for each preset personality type, multiple sample question-answer pairs are used to train a basic model to obtain a personality model corresponding to the preset personality type, wherein the output distribution of the personality model is aligned with the first sample replies; the personality model is used to generate a scenario reply corresponding to a scenario question, and a preference score of the personality model is obtained based on the scenario reply, wherein the preference score is positively correlated with human preference, and the scenario question is a question under a specified scenario; based on the preference score of each personality model, all personality models are merged into a target question-answer model under the specified scenario.

[0072] In this embodiment, a scenario question answering model generation method is provided, such as Figure 2 As shown, the specific process of the scenario question-answering model generation method can be as follows:

[0073] S110, obtaining multiple sample question-answer pairs corresponding to each preset personality type.

[0074] The preset personality type refers to the personality obtained by classifying human beings according to common personality characteristics. Under different personality classification systems, the number and meaning of preset personality types are different. For each preset personality type, multiple sample question-answer pairs corresponding to the preset personality type can be obtained.

[0075] The sample question-answer pair may include a sample question, a first sample answer, and a second sample answer, wherein the first sample answer and the second sample answer are both answers corresponding to the sample question. Users with different personalities may give different answers to the same question, wherein the first sample answer is an answer that matches the preset personality type, and the second sample answer is an answer that does not match the preset personality type.

[0076] Multiple sample question-answer pairs for each preset personality type can be collected in advance, and then the multiple sample question-answer pairs corresponding to each preset personality type can be stored in a database. When needed, the multiple sample question-answer pairs corresponding to each preset personality type can be directly obtained from the database.

[0077] In some embodiments, when obtaining multiple sample question-answer pairs corresponding to each preset personality type, it can be: obtaining a basic dialogue data set, the basic dialogue data set includes multiple sample questions; for each of the sample questions, obtaining multiple sample replies corresponding to the sample question through a basic model, wherein one sample reply corresponds to a preset personality type; for each of the sample replies, calculating the output value corresponding to the sample reply; using the output value corresponding to each sample reply, calculating the output difference of each sample pair, the sample pair includes a first sample reply and a second sample reply that meet a preset condition, the output difference being the difference between the output value of the first sample reply and the output value of the second sample reply; if the output difference is greater than a preset threshold, combining the first sample reply, the second sample reply, and the sample question as a sample question-answer pair of the preset personality type corresponding to the first sample reply and the second sample reply.

[0078] The basic dialogue dataset is composed of multiple question-answer pairs, each of which can include a sample question and the corresponding answer to the sample question. These question-answer pairs can be question-answer pairs in a single round of dialogue or in multiple rounds of dialogue. The basic dialogue dataset can be compiled from various open source data, network data, books, and dialogue business system data. Obviously, the question-answer pairs in the basic dialogue dataset are general question-answer pairs.

[0079] Each question-answer pair in the basic dialogue dataset contains at least one sample question, and the sample questions can be obtained from the basic dialogue dataset for subsequent use. The basic model can be a general large language model with basic reasoning and question-answering capabilities. For each sample question, the basic model can be used to infer the sample question to obtain the corresponding sample answer, where the sample answers for the same sample question are different under different preset personality types. The basic model can obtain multiple sample answers, and one sample answer corresponds to one preset personality type. Therefore, for a sample question, multiple sample answers corresponding to the sample question can be inferred, and the number of sample answers is consistent with the number of preset personality types.

[0080] As an implementation method, a sample question and a corresponding prompt word of a preset personality type can be input into a basic model to obtain a sample answer under the corresponding preset personality type. The prompt word and sample question of each preset personality type are respectively input into the basic model to obtain multiple sample answers corresponding to each preset personality type.

[0081] As an implementation method, when obtaining multiple sample answers corresponding to sample questions through a basic model, it can be: for each of the sample questions, generating a personality template corresponding to each preset personality type based on a specified prompt word template, the personality template including a personality description corresponding to the preset personality type; combining the sample question with each of the personality templates respectively to obtain multiple sample inputs corresponding to the sample question; for each of the sample inputs, inferring the sample input through the basic model to obtain a sample answer matching each of the preset personality types.

[0082] The designated prompt template is a predefined general prompt word template, and the designated prompt template may include description information corresponding to each personality factor. Among them, personality factors are key factors affecting personality in the preset personality classification system, and different preset personalities are composed of different personality factors. The preset personality classification system is a classification system for preset personality types, and personality factors may be different under different personality classification systems.

[0083] For example, in the Myers-Briggs Type Indicator (MBTI), personality is divided into four dimensions, each of which consists of two opposing directions. The life attitude dimension includes two directions: extroversion E and introversion I; the perception dimension includes two directions: feeling S and intuition N; the judgment dimension includes two directions: thinking T and feeling F; the lifestyle dimension includes two directions: judgment J and perception P. Among them, E, I, S, N, T, F, J, and P are personality factors. Selecting a personality factor in each dimension can combine to obtain 16 different preset personality types.

[0084] It should be noted that the personality classification system can also be Enneagram, DISC, Big Five, etc., which can be set according to actual needs. The following explanation will only use MBTI as an example.

[0085] The specified prompt word template may also include a personality type slot, a personality type description slot, and a question slot. For example, taking MBTI as an example, the specified prompt word template may be:

[0086] ###Background Knowledge###

[0087] MBTI's 16 personality types are described according to the following four dimensions:

[0088] 1. How we interact with the world

[0089] E Extraversion focuses on how you affect your external environment: focusing your mental energy and attention on the external world and getting along with other people.

[0090] I Introverts focus on the impact of changes in the external environment on themselves: they focus their psychological energy and attention on the internal world and pay attention to their own inner feelings.

[0091] 2. Main ways to obtain information

[0092] S Sensation Focus on specific information obtained by sensory organs: see, hear, smell, taste, touch things, pay attention to details, like to describe, like to use and think about known skills

[0093] N Intuition focuses on the overall and development trends of things: inspiration, prediction, suggestion and reasoning. Pay attention to imagination and originality, like to learn new skills, but get bored easily, like to describe, and present facts in a jumpy manner.

[0094] 3. Decision-making method

[0095] T values ​​the logical relationship between things and likes to make decisions and evaluations through objective analysis. Rational, objective, fair

[0096] F Emotions are based mainly on one's own and others' feelings, and values ​​are the criteria for judgment.

[0097] 4. Ways of doing things

[0098] J judges that they like to make plans and decisions, are willing to manage and control, want their lives to be orderly, pay attention to results, follow rules, be organized, and respect time limits.

[0099] P Flexible perception, trying to understand, adapt to the environment, focus on leaving space, like a relaxed and free lifestyle. Pay attention to the process, constantly adjust the goal as the data changes, and like to have a variety of options.

[0100] ###instruction###

[0101] You have to play a person with {character_type} personality, always keep in mind the description of sex and character characteristics, and complete the dialogue

[0102] ###Personality traits###

[0103] {character_description}

[0104] ###question###

[0105] {question}

[0106] The background knowledge includes the description of each personality factor. {character_type} is the personality type slot, {character_description} is the personality type description slot, and {question} is the question slot.

[0107] For each preset personality type, the preset personality type is filled into the personality type slot, and the personality description corresponding to the preset personality type is filled into the personality type description slot, so that the personality template corresponding to the preset personality type can be obtained.

[0108] Since the specified prompt word template includes a question slot, and the question slot is not processed when the personality template is generated, the personality template includes the question slot, and the sample question is filled into the question slot of each personality template to obtain the sample input corresponding to each preset personality type.

[0109] Thus, a sample question can generate multiple sample inputs, and one sample input corresponds to one preset personality type. The sample question corresponding to each sample input is provided to the basic model, and the basic model analyzes and infers it to obtain a sample answer corresponding to each sample input, and the sample answer matches the preset personality type in the sample input.

[0110] In order to measure the differences in responses to different preset personality types, for each sample response to each sample question, the output value corresponding to the sample response can be calculated, and then the output difference can be calculated using the output value to determine the first sample response and the second sample response corresponding to the sample question.

[0111] As an implementation method, when calculating the output value corresponding to the sample answer, it can be that for each sample answer, the original score matrix generated when the basic model predicts the sample answer is obtained; for each row in the original score matrix, the column index corresponding to the sample answer and the maximum original score in the row are obtained; the difference between the score corresponding to the column index in the original score matrix and the maximum original score is calculated; based on the difference, the output value corresponding to the sample answer is calculated.

[0112] Among them, the raw score matrix is ​​the logits part corresponding to the sample response output by the basic model, which is a . Among them, n is the total length of the word unit in the sample answer, the word unit is the token, and v refers to the vocabulary length of the basic model. The output sample answer is obtained by selecting a word unit in each row in the original score matrix. That is, each row in the original score matrix represents the logits part of a token, and the token is one of the words in the vocabulary with a length of v. The column index corresponding to the sample answer is the index of the vocabulary finally selected by the token. For example, if the row index is 1 and the column index is 3, the first token selects the third word as the output. For each row in the original score matrix, the maximum original score in the row can be obtained. Subtract the maximum original score from the score at the column index in the corresponding row to get the difference. Finally, the difference can be used to calculate the output value of the sample answer.

[0113] Specifically, the output value of the sample response can be calculated according to the following formula:

[0114] ;

[0115] Among them, R represents the output value of the sample response; X represents the corresponding The raw score matrix of , where n is the total length of the output word unit and v is the length of the vocabulary of the basic model; In the raw score matrix representing the sample response, the raw score corresponding to the ith row and idth column, where id is the subscript of the output word unit in the vocabulary; represents the maximum raw score in the i-th row; Represents the jth raw score in the i-th row.

[0116] After calculating the output value corresponding to each sample response, the output difference between each sample pair can be calculated. The sample pair consists of a first sample response and a second sample response that meet the preset conditions, and the output difference is obtained by subtracting the output value of the second sample response from the output value of the first sample response. Optionally, when calculating the output difference of the sample pair, for each of the sample questions, a response sequence composed of multiple sample responses corresponding to the sample question can be obtained; based on the index of each sample response in the response sequence, multiple sample pairs can be determined; the output value of the first sample response in the sample pair is used to subtract the output value of the second sample response to obtain the output difference of the sample pair.

[0117] Among them, multiple sample responses corresponding to the sample question can form a response sequence, and the specific sorting method can be set according to actual needs and will not be specifically limited here. Each sample response has its corresponding index in the response sequence, and each sample response can be denoted as A k , and the maximum value of k is the number of preset personality types.

[0118] When determining the sample pair, it is necessary to determine the first response sample and the second response sample. If the first response sample is denoted as A k1 , and the second response sample is denoted as A k2 , then the preset condition k1 < k2 needs to be satisfied. In this way, multiple sample pairs can be combined. For each sample pair, the output value of the first sample response in it is used to subtract the output value of the second sample response to obtain the output difference.

[0119] If the output difference is greater than the preset threshold, it indicates that the difference between the two sample responses is large, that is, the difference between the preset personality types corresponding to these two sample responses on the same question is large. In order to better distinguish different preset personality types, the first sample response, the second sample response, and the sample question can be combined into a sample question-and-answer pair.

[0120] Optionally, if the output difference is greater than the preset threshold, when combining the first sample response, the second sample response, and the sample question into a sample question-and-answer pair, it can be that if the output difference is greater than the preset threshold, the sample question, the first sample response, and the second sample response are combined in sequence as the sample question-and-answer pair corresponding to the preset personality type of the first sample response; the sample question, the second sample response, and the first sample response are combined in sequence as the sample question-and-answer pair corresponding to the preset personality type of the second sample response.

[0121] Combining the sample question, the first sample response, and the second sample response in sequence can obtain a sample question-and-answer pair, which can be denoted as the first sample question-and-answer pair for the convenience of description. Combining the sample question, the second sample response, and the first sample response in sequence can obtain a sample question-and-answer pair, denoted as the second sample question-and-answer pair.

[0122] Record the preset personality type corresponding to the first sample response as the first personality, and record the preset personality type corresponding to the second sample response as the second personality. The first sample Q&A pair is the sample Q&A pair corresponding to the first personality, and the second sample Q&A pair is the sample Q&A pair corresponding to the second personality.

[0123] For example, sample question Q, first sample response A k1 is the response corresponding to personality type 1, and second response sample A k2 is the response corresponding to personality type 2, k1 < k2, and the output difference is greater than the preset threshold. Then the sample Q&A pair of personality type 1 is (Q, A k1 , A k2 ), and the sample Q&A pair of personality type 2 is (Q, A k2 , A k1 ).

[0124] Traverse each sample question in the basic dialogue dataset in the above manner to obtain multiple sample Q&A pairs corresponding to each preset personality type.

[0125] S120. For each of the preset personality types, use the multiple sample Q&A pairs to train the basic model to obtain a personality model corresponding to the preset personality type.

[0126] To obtain a personality model corresponding to each preset personality type, the basic model can be trained using multiple sample Q&A pairs corresponding to the preset personality type. In each of the foregoing sample Q&A pairs, a first sample response that matches the preset personality type and a second sample response that does not match the preset personality type are included. To ensure that the content output by the personality model is more in line with the preset personality type, the training objective can be to align the output distribution of the personality model with the first sample response. The style of the content output by the trained personality model matches its corresponding preset personality type.

[0127] As an implementation method, specifically when training the personality models corresponding to each preset personality type, the basic model may be copied for each preset personality type to obtain a reference model; the sample question is input into the basic model to obtain a first predicted probability of generating the first sample answer and a second predicted probability of generating the second sample answer; the sample question is input into the reference model to obtain a first reference probability of generating the first sample answer and a second reference probability of generating the second sample answer; a loss function is constructed based on the first predicted probability, the second predicted probability, the first reference probability and the second reference probability; if the loss value of the loss function does not meet the preset convergence condition, the model parameters of the basic model are adjusted; if the loss value of the loss function meets the preset convergence condition, the personality model corresponding to the preset personality type is obtained.

[0128] For each preset personality type, multiple sample question-answer pairs corresponding to it are used for training. During training, the base model needs to be copied to obtain a reference model. The base model will update its model parameters during training, while the model parameters of the reference model will not be updated, in order to provide a stable baseline for the base model to prevent it from deviating too far from the original behavior during training.

[0129] Input the sample question in the sample question-answer pair into the basic model, and obtain the first predicted probability corresponding to the first sample answer output by the basic model, and the second predicted probability corresponding to the second sample answer. Similarly, input the sample question into the reference model, and obtain the first reference probability of the first sample answer output by the reference model, and the second reference probability of the second sample answer output by the reference model;

[0130] Then, using these four probabilities, we can construct a loss function, which is as follows:

[0131] ;

[0132] in, represents the loss function; D represents multiple sample question-answer pairs; E represents the average of all sample question-answer pairs; Characterizing the first prediction probability; characterizing a second prediction probability; Characterizing a first reference probability; characterizing a second reference probability; is the sigmoid function; is a hyperparameter.

[0133] During the training process, the loss value corresponding to the loss function can be calculated. When the loss value does not meet the preset convergence condition, the gradient descent method can be used to update and adjust the model parameters of the basic model so that the loss value can meet the preset convergence condition. The preset convergence condition can be that the loss value no longer decreases or is lower than a predetermined value. When the loss value of the loss function meets the preset convergence condition, the training can be considered to be completed, and the model parameters of the model are no longer updated. The currently acquired model parameters are used as the model parameters of the personality model, thereby obtaining a personality model. Among them, the style of the response generated by the trained personality model is consistent with the corresponding preset personality type.

[0134] S130: Generate a scenario answer corresponding to the scenario question using the personality model, and obtain a preference score of the personality model based on the scenario answer.

[0135] Scenario questions refer to questions that users may ask in a specified scenario. For example, in a traditional Chinese medicine scenario, a scenario question may be: What are the effects of Sedum? The specified scenario can be determined based on actual project requirements. Each trained personality model can analyze and reason about scenario questions to generate corresponding scenario responses. The preference score is a score that is positively correlated with human preferences and is used to measure the fit between scenario responses and human preferences. The more the scenario responses generated by the personality model conform to human preferences, the higher the preference score of the personality model.

[0136] As an implementation method, when obtaining the preference score of a personality model, a scenario question set under a specified scenario may be obtained, wherein the scenario question set includes multiple scenario questions; two personality models are randomly selected from the multiple personality models as the first personality model and the second personality model, and a specified question is randomly selected from the scenario question set; the specified question is input into the first personality model and the second personality model respectively to obtain a first specified answer corresponding to the first personality model and a second specified answer corresponding to the second personality model; based on the comparison result of the first specified answer and the second specified answer, the preference scores of the first personality model and the second personality model are updated until the number of comparisons is greater than a preset number, and the preference score of each personality model is obtained.

[0137] See also Figure 3 , which shows a flow chart of obtaining preference scores of each personality model. The scenario question set may include multiple scenario questions, and a scenario question may be randomly sampled from the scenario question set as a designated question. Similarly, two personality models may be randomly selected from multiple personality models as the first personality model and the second personality model.

[0138] Then, the designated question is input into the first personality model and the second personality model respectively, and a first designated answer output by the first personality model and a second designated answer output by the second personality model can be obtained. Then, the first designated answer and the second designated answer can be compared to obtain a comparison result. As an implementation mode, when comparing the first designated answer and the second designated answer, the first designated answer and the second designated answer can be scored according to a preset comparison rule, so as to determine the comparison result based on the scoring result. As another implementation mode, it is possible to manually determine which of the first designated answer and the second designated answer is better to obtain a comparison result, for example, manually mark the better answer of the two answers, and obtain the comparison result by identifying the mark.

[0139] The preference scores of the first personality model and the second personality model can then be updated based on the comparison results. As an implementation method, the same initial score can be set for each personality model, and the score update rules can be pre-set. For example, when comparing the designated answers corresponding to the two personality models, the score of the personality model corresponding to the selected designated answer is increased by S1, and the score of the personality model corresponding to the unselected designated answer is increased by S2. Among them, S1 and S2 can be set according to actual needs, and they can be positive or negative. Thus, by randomly extracting models and designated questions multiple times and comparing the designated answers of the two personality models, the initial scores of the two extracted personality models can be updated to obtain preference scores.

[0140] During the comparison process, the number of comparisons can be counted, and each time a comparison is performed, 1 is added to the original number of comparisons to obtain the latest number of comparisons. If the number of comparisons is not greater than the preset number, the process can return to continue randomly extracting the specified question, the first personality model, and the second personality model, and continuously update the preference score of the corresponding personality model with the comparison results. If the number of comparisons is greater than the preset number, the comparison can be considered to be completed, and the current preference scores of all personality models can be obtained for subsequent use. The preference scores can reflect the user's preference in the specified scenario. It should be noted that the data volume of the scenario question set in the specified scenario is much smaller than the data volume of the basic dialogue set, that is, a large amount of general data is used to achieve model differentiation, and a small amount of scenario data is used to fuse the model, which is convenient for application in a variety of different scenarios.

[0141] S140: Based on the preference score of each personality model, all personality models are integrated into a target question-answering model in the specified scenario.

[0142] After obtaining the preference score of each personality model, the preference score can be used to fuse all personality models into a target question-answering model. Since the preference score is evaluated based on scenario questions in a specified scenario, the preference score can reflect the user's preference in the specified scenario. Fusion of multiple personality models based on the preference score to obtain the target question-answering model can ensure that the performance of the final fused target question-answering model in the specified scenario meets the user's preference.

[0143] As an implementation method, when a target question-answering model is obtained by fusing multiple personality models based on preference scores, it can be: for each personality model, the difference between the preference scores of other personality models and the preference score of the personality model is used to calculate the score to be used of the personality model; the scores to be used corresponding to all the personality models are normalized to obtain the fusion weight corresponding to each personality model; and according to the fusion weight corresponding to each personality model, all personality models are fused into the target question-answering model for the specified scenario.

[0144] The score to be used is the score directly used when fusing multiple personality models, and can be calculated by the difference between the preference scores of the personality model and other personality models. Optionally, for each personality model, the personality model is determined as the target personality model, and other personality models other than the target personality model are determined as candidate personality models; the difference between the preference scores of each candidate personality model and the target personality model is calculated; and the score to be used corresponding to the target personality model is calculated using the difference.

[0145] That is, each personality model can be used as a target personality model, and other personality models are candidate personality models. The preference score of each candidate personality model is subtracted from the preference score of the target personality model to obtain the difference, and the difference is used to calculate the score to be used. Specifically, the score to be used of each personality model can be calculated according to the following formula:

[0146] ;

[0147] Among them, W i The score to be used that represents the target personality model; R j Characterize the preference scores of other personality models; R i A preference score that characterizes the target personality model.

[0148] After calculating the scores to be used corresponding to each personality model, all the scores to be used may be normalized to obtain the fusion weights of each personality model. The normalization may be performed by processing all the scores to be used using a softmax function to obtain the fusion weights.

[0149] In some embodiments, when fusing multiple personality models to obtain a target question-answering model, the steps may be to obtain a model parameter vector corresponding to each personality model; determine a first model to be fused and a second model to be fused from the multiple personality models; fuse the model parameter vector of the first model to be fused and the parameter vector of the second model to be fused using the fusion weight of the first model to be fused and the fusion weight of the second model to be fused to obtain an intermediate fusion vector; determine the intermediate fusion vector as the model parameter vector of the first model to be fused, and determine a new second model to be fused from the remaining personality models; return to execute the step of fusing the model parameter vector of the first model to be fused and the parameter vector of the second model to be fused using the fusion weight of the first model to be fused and the fusion weight of the second model to be fused to obtain an intermediate fusion vector, until all the personality models are traversed; if all the personality models are traversed, the target question-answering model in the specified scenario is obtained based on the intermediate fusion vector.

[0150] Each personality model has its corresponding model parameter vector, and fusing the personality models is essentially fusing the model parameter vectors. First, a first model to be fused and a second model to be fused are determined from multiple personality models, and then the corresponding model parameter vectors are fused using the fusion weights of the first model to be fused and the second model to be fused to obtain an intermediate model parameter vector.

[0151] Optionally, the intermediate model parameter vector may be calculated according to the following formula:

[0152] ;

[0153] Among them, V m represents the intermediate model parameter vector; V1 represents the model parameter vector of the first model to be fused; V2 represents the model parameter vector of the first model to be fused; θ represents the angle between the model parameter vector of the first model to be fused and the model parameter vector of the second model to be fused; t is the interpolation factor, which is related to the fusion weight of the first model to be fused and the fusion weight of the second model to be fused.

[0154] Among them, if the fusion weight of the first model to be fused is A1, and the fusion weight of the second model to be fused is B1, the interpolation factor can be t = B1 / (A1 + B1)). After calculating the intermediate fusion vector, the intermediate fusion vector can be further fused with the model parameter vectors of other personality models until the model parameter vectors of all personality models are fused. It should be noted that the intermediate fusion vector is subsequently determined as the model parameter vector of the first model to be fused. At this time, the fusion weight of the first model to be fused can be considered as the sum of the fusion weights of the first model to be fused and the second model to be fused.

[0155] The new intermediate fusion vector is calculated in the above manner, and the cycle continues until the model parameter vectors of all personality models are fused to obtain the final intermediate fusion vector. The final intermediate fusion vector is the model parameter vector of the final target question-answering model. The output of the target question-answering model in the specified scenario is more in line with user preferences, thereby improving the question-answering effect in the specified scenario.

[0156] The scenario question-answering model generation scheme provided by the embodiment of the present invention can be applied to question-answering scenarios in various specific scenarios. For example, taking the TCM health care scenario as an example, the scheme provided by the embodiment of the present invention can use a small number of questions in the TCM health care scenario, and integrate multiple personality models to obtain the target question-answering model in the TCM health care scenario. When processing questions in the TCM health care scenario, the target question-answering model can give answers that are more in line with user preferences, thereby improving the naturalness of the question-answering and user satisfaction.

[0157] The method provided by the embodiment of the present invention can use multiple sample question-answer pairs corresponding to the preset personality type to train the basic model to obtain the personality model corresponding to the preset personality type, so that the output of each personality model matches the preset personality type. The preference score of the personality model is then calculated using the response of the personality model in the specified scenario, and finally all personality models are fused according to the preference score to obtain the target question-answer model, which can improve the question-answer quality of the target question-answer model in the specified scenario. When generating the target question-answer model in the specified scenario, the amount of data of the multiple sample question-answer pairs used is much larger than the amount of data in the specified scenario, which can achieve rapid adaptation and deployment of the target question-answer model in various specified scenarios.

[0158] In order to better implement the above method, the embodiment of the present invention also provides a scenario question-answering model generation device, which can be integrated in an electronic device, and the electronic device can be a terminal, a server, etc. Among them, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0159] For example, in this embodiment, the method of the embodiment of the present invention is described in detail by taking the scenario question and answer model generation device being specifically integrated in a server as an example.

[0160] For example, Figure 4 As shown, the scenario question and answer model generation device 200 may include an acquisition module 210, a training module 220, a scenario processing module 230 and a fusion module 240.

[0161] An acquisition module 210 is used to acquire a plurality of sample question-answer pairs corresponding to each preset personality type, wherein the sample question-answer pairs include a sample question, a first sample answer, and a second sample answer, wherein the first sample answer matches the preset personality type, and the second sample answer does not match the preset personality type;

[0162] A training module 220, for training a basic model using the plurality of sample question-answer pairs for each of the preset personality types, to obtain a personality model corresponding to the preset personality type, wherein an output distribution of the personality model is aligned with the first sample answer;

[0163] A scenario processing module 230, for generating a scenario answer corresponding to a scenario question using the personality model, and obtaining a preference score of the personality model based on the scenario answer, wherein the preference score is positively correlated with human preference, and the scenario question is a question under a specified scenario;

[0164] The fusion module 240 is used to fuse all personality models into a target question-answering model in the specified scenario based on the preference score of each personality model.

[0165] In some embodiments, the acquisition module 210 is specifically used to:

[0166] Acquire a basic dialogue data set, wherein the basic dialogue data set includes a plurality of sample questions;

[0167] For each of the sample questions, a plurality of sample answers corresponding to the sample question are obtained through the basic model, wherein one sample answer corresponds to one preset personality type;

[0168] For each of the sample responses, calculating an output value corresponding to the sample response;

[0169] Calculate the output difference of each sample pair with the output value corresponding to each sample answer, wherein the sample pair includes a first sample answer and a second sample answer that meet a preset condition, and the output difference is the difference between the output value of the first sample answer and the output value of the second sample answer;

[0170] If the output difference is greater than a preset threshold, the first sample answer, the second sample answer, and the sample question are combined as a sample question-answer pair of a preset personality type corresponding to the first sample answer and the second sample answer.

[0171] In some embodiments, the acquisition module 210 is specifically used to:

[0172] For each sample answer, obtain the original score matrix generated when the base model predicts the sample answer;

[0173] For each row in the raw score matrix, obtain the column index corresponding to the sample answer and the maximum raw score in the row;

[0174] Calculate the difference between the score corresponding to the column index in the raw score matrix and the maximum raw score;

[0175] Based on the difference, an output value corresponding to the sample response is calculated.

[0176] In some embodiments, the acquisition module 210 is specifically used to:

[0177] If the output difference is greater than a preset threshold, combining the sample question, the first sample answer, and the second sample answer in sequence as a sample question-answer pair of a preset personality type corresponding to the first sample answer;

[0178] The sample question, the second sample answer and the first sample answer are combined in sequence as a sample question-answer pair of a preset personality type corresponding to the second sample answer.

[0179] In some embodiments, the acquisition module 210 is specifically used to:

[0180] For each of the sample questions, generating a personality template corresponding to each preset personality type based on a designated prompt word template, wherein the personality template includes a personality description corresponding to the preset personality type;

[0181] Combining the sample question with each of the personality templates to obtain a plurality of sample inputs corresponding to the sample question;

[0182] For each of the sample inputs, the sample input is inferred through the basic model to obtain a sample response that matches each of the preset personality types.

[0183] In some embodiments, the training module 220 is specifically used to:

[0184] For each of the preset personality types, the basic model is copied to obtain a reference model;

[0185] Inputting the sample question into the basic model to obtain a first predicted probability of generating the first sample answer and a second predicted probability of generating the second sample answer;

[0186] Inputting the sample question into the reference model to obtain a first reference probability of generating the first sample answer and a second reference probability of generating the second sample answer;

[0187] constructing a loss function based on the first predicted probability, the second predicted probability, the first reference probability, and the second reference probability;

[0188] If the loss value of the loss function does not meet the preset convergence condition, adjusting the model parameters of the basic model;

[0189] If the loss value of the loss function satisfies the preset convergence condition, a personality model corresponding to the preset personality type is obtained.

[0190] In some embodiments, the scene processing module 230 is specifically used to:

[0191] Obtain a scenario problem set under a specified scenario, wherein the scenario problem set includes multiple scenario problems;

[0192] randomly extracting two personality models from the plurality of personality models as a first personality model and a second personality model, and randomly extracting a designated question from the scenario question set;

[0193] inputting the designated question into the first personality model and the second personality model respectively, obtaining a first designated answer corresponding to the first personality model and a second designated answer corresponding to the second personality model;

[0194] Based on the comparison result of the first designated answer and the second designated answer, the preference scores of the first personality model and the second personality model are updated until the number of comparisons is greater than a preset number, and the preference score of each personality model is obtained.

[0195] In some embodiments, the fusion module 240 is specifically used to:

[0196] For each personality model, using the difference between the preference scores of other personality models and the preference score of the personality model, calculate the score to be used of the personality model;

[0197] Normalizing the to-be-used scores corresponding to all the personality models to obtain a fusion weight corresponding to each personality model;

[0198] According to the fusion weight corresponding to each personality model, all personality models are fused into the target question-answering model for the specified scenario.

[0199] In some embodiments, the fusion module 240 is specifically used to:

[0200] Obtaining a model parameter vector corresponding to each personality model;

[0201] Determining a first model to be fused and a second model to be fused from the multiple personality models;

[0202] Using the fusion weight of the first model to be fused and the fusion weight of the second model to be fused, the model parameter vector of the first model to be fused and the model parameter vector of the second model to be fused are fused to obtain an intermediate fusion vector;

[0203] Determining the model parameter vector of the first model to be fused by using the intermediate fusion vector, and determining a new second model to be fused from the remaining personality models;

[0204] Returning to the step of fusing the model parameter vector of the first model to be fused and the model parameter vector of the second model to be fused using the fusion weight of the first model to be fused and the fusion weight of the second model to be fused to obtain an intermediate fusion vector, until all the personality models are traversed;

[0205] If all the personality models are traversed, the target question-answering model in the specified scenario is obtained based on the intermediate fusion vector.

[0206] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.

[0207] As can be seen from the above, the scenario question-answering model generation device of this embodiment can use multiple sample question-answer pairs corresponding to each preset personality type to train the basic model, obtain the personality model corresponding to the preset personality type, so that the output of each personality model matches the preset personality type. Then, the preference score of the personality model is calculated using the personality model's response in the specified scenario, so as to measure whether the personality model's response in the specified scenario conforms to human preferences. Finally, all personality models are fused according to the preference score to obtain the target question-answering model, so that the target question-answering model can give answers that are more in line with user preferences when answering questions in the specified scenario, which can effectively improve the naturalness of the conversation of the question-answering model in the specified scenario.

[0208] The embodiment of the present invention further provides an electronic device, which may be a terminal, a server, or the like. The terminal may be a mobile phone, a tablet computer, a smart Bluetooth device, a notebook computer, a personal computer, or the like; the server may be a single server or a server cluster composed of multiple servers, or the like.

[0209] In some embodiments, the scenario question and answer model generation device can also be integrated into multiple electronic devices. For example, the scenario question and answer model generation device can be integrated into multiple servers, and the scenario question and answer model generation method of the present invention can be implemented by multiple servers.

[0210] In this embodiment, the electronic device of this embodiment is a server as an example for detailed description, for example, Figure 5 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present invention, specifically:

[0211] The electronic device may include components such as a processor 310 with one or more processing cores, a memory 320 with one or more computer-readable storage media, a power supply 330, an input module 340, and a communication module 350. Those skilled in the art will appreciate that Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0212] The processor 310 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 320, and calling data stored in the memory 320. In some embodiments, the processor 310 may include one or more processing cores; in some embodiments, the processor 310 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 310.

[0213] The memory 320 can be used to store software programs and modules. The processor 310 executes various functional applications and data processing by running the software programs and modules stored in the memory 320. The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 320 may also include a memory controller to provide the processor 310 with access to the memory 320.

[0214] The electronic device also includes a power supply 330 for supplying power to various components. In some embodiments, the power supply 330 can be logically connected to the processor 310 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 330 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0215] The electronic device may further include an input module 340, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0216] The electronic device may further include a communication module 350. In some embodiments, the communication module 350 may include a wireless module. The electronic device may perform short-range wireless transmission through the wireless module of the communication module 350, thereby providing the user with wireless broadband Internet access. For example, the communication module 350 may be used to help the user send and receive emails, browse web pages, and access streaming media.

[0217] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 310 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 320 according to the following instructions, and the processor 310 will run the application programs stored in the memory 320, thereby implementing the steps in the methods of the embodiments of the present invention.

[0218] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0219] As can be seen from the above, the electronic device provided by the embodiment of the present invention can use multiple sample question-answer pairs corresponding to each preset personality type to train the basic model, obtain the personality model corresponding to the preset personality type, so that the output of each personality model matches the preset personality type. Then, the preference score of the personality model is calculated using the personality model's response in the specified scenario, so as to measure whether the personality model's response in the specified scenario conforms to human preferences. Finally, all personality models are fused according to the preference score to obtain the target question-answer model, so that the target question-answer model can give answers that are more in line with user preferences when answering questions in the specified scenario, which can effectively improve the naturalness of the conversation of the question-answer model in the specified scenario.

[0220] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0221] To this end, an embodiment of the present invention provides a computer-readable storage medium, in which a plurality of instructions are stored. The instructions can be loaded by a processor to execute the steps in any method provided by the embodiment of the present invention.

[0222] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0223] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including a computer program / instruction, the computer program / instruction being stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instruction from the computer-readable storage medium, and the processor executes the computer program / instruction, so that the electronic device executes the method provided in various optional implementations of the scene question-answering model generation or model training provided in the above-mentioned embodiments.

[0224] Since the instructions stored in the storage medium can execute the steps in any scenario question and answer model generation method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any scenario question and answer model generation method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0225] The above is a detailed introduction to a scenario question and answer model generation method and device provided in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A scenario question answering model generation method, characterized in that: The method comprises: Acquire multiple sample question-answer pairs corresponding to each preset personality type, wherein the sample question-answer pairs include a sample question, a first sample answer, and a second sample answer, wherein the first sample answer matches the preset personality type, and the second sample answer does not match the preset personality type; For each of the preset personality types, use the multiple sample question-answer pairs to train a basic model to obtain a personality model corresponding to the preset personality type, wherein an output distribution of the personality model is aligned with the first sample answer; Generate a scenario answer corresponding to a scenario question using the personality model, and obtain a preference score of the personality model based on the scenario answer, wherein the preference score is positively correlated with human preference, and the scenario question is a question under a specified scenario; Based on the preference score of each personality model, all personality models are integrated into a target question-answering model for the specified scenario; The step of obtaining a plurality of sample question-answer pairs corresponding to each preset personality type includes: A basic dialogue data set is obtained, wherein the basic dialogue data set includes a plurality of sample questions; for each of the sample questions, a personality template corresponding to each preset personality type is generated based on a specified prompt word template, wherein the personality template includes a personality description corresponding to the preset personality type; the sample question is respectively combined with each of the personality templates to obtain a plurality of sample inputs corresponding to the sample question; for each of the sample inputs, the sample input is inferred through the basic model to obtain a sample reply matching each of the preset personality types; for each of the sample replies, an output value corresponding to the sample reply is calculated; with the output value corresponding to each sample reply, an output difference value of each sample pair is calculated, wherein the sample pair includes a first sample reply and a second sample reply that meet a preset condition, and the output difference value is the difference between the output value of the first sample reply and the output value of the second sample reply; if the output difference value is greater than a preset threshold value, the first sample reply, the second sample reply and the sample question are combined as a sample question-answer pair of the preset personality type corresponding to the first sample reply and the second sample reply.

2. The method according to claim 1, characterized in that The step of calculating, for each of the sample responses, an output value corresponding to the sample response, comprises: For each sample answer, obtain the original score matrix generated when the base model predicts the sample answer; For each row in the raw score matrix, obtain the column index corresponding to the sample answer and the maximum raw score in the row; Calculate the difference between the score corresponding to the column index in the raw score matrix and the maximum raw score; Based on the difference, an output value corresponding to the sample response is calculated.

3. The method according to claim 1, characterized in that If the output difference is greater than a preset threshold, combining the first sample answer, the second sample answer, and the sample question as a sample question-answer pair of a preset personality type corresponding to the first sample answer and the second sample answer includes: If the output difference is greater than a preset threshold, combining the sample question, the first sample answer, and the second sample answer in sequence as a sample question-answer pair of a preset personality type corresponding to the first sample answer; The sample question, the second sample answer and the first sample answer are combined in sequence as a sample question-answer pair of a preset personality type corresponding to the second sample answer.

4. The method according to claim 1, characterized in that: For each of the preset personality types, the basic model is trained using the multiple sample question-answer pairs to obtain a personality model corresponding to the preset personality type, including: For each of the preset personality types, the basic model is copied to obtain a reference model; Inputting the sample question into the basic model to obtain a first predicted probability of generating the first sample answer and a second predicted probability of generating the second sample answer; Inputting the sample question into the reference model to obtain a first reference probability of generating the first sample answer and a second reference probability of generating the second sample answer; constructing a loss function based on the first predicted probability, the second predicted probability, the first reference probability, and the second reference probability; If the loss value of the loss function does not meet the preset convergence condition, adjusting the model parameters of the basic model; If the loss value of the loss function satisfies the preset convergence condition, a personality model corresponding to the preset personality type is obtained.

5. The method according to claim 1, characterized in that The step of generating a scenario answer corresponding to a scenario question by using the personality model, and obtaining a preference score of the personality model based on the scenario answer, includes: Obtain a scenario problem set under a specified scenario, wherein the scenario problem set includes multiple scenario problems; Randomly extract two personality models from a plurality of personality models as a first personality model and a second personality model, and randomly extract a designated question from the scenario question set; inputting the designated question into the first personality model and the second personality model respectively, obtaining a first designated answer corresponding to the first personality model and a second designated answer corresponding to the second personality model; Based on the comparison result of the first designated answer and the second designated answer, the preference scores of the first personality model and the second personality model are updated until the number of comparisons is greater than a preset number, and the preference score of each personality model is obtained.

6. The method according to claim 1, characterized in that The step of fusing all personality models into a target question-answering model in the specified scenario based on the preference score of each personality model includes: For each of the personality models, using the difference between the preference scores of other personality models and the preference score of the personality model, calculate the score to be used of the personality model; Normalizing the to-be-used scores corresponding to all the personality models to obtain a fusion weight corresponding to each personality model; According to the fusion weight corresponding to each personality model, all personality models are fused into the target question-answering model for the specified scenario.

7. The method according to claim 6, characterized in that According to the fusion weight corresponding to each personality model, all personality models are fused into a target question-answering model in a specified scenario, including: Obtaining a model parameter vector corresponding to each personality model; Determine a first model to be fused and a second model to be fused from a plurality of personality models; Using the fusion weight of the first model to be fused and the fusion weight of the second model to be fused, the model parameter vector of the first model to be fused and the model parameter vector of the second model to be fused are fused to obtain an intermediate fusion vector; Determining the model parameter vector of the first model to be fused by using the intermediate fusion vector, and determining a new second model to be fused from the remaining personality models; Returning to the step of fusing the model parameter vector of the first model to be fused and the model parameter vector of the second model to be fused using the fusion weight of the first model to be fused and the fusion weight of the second model to be fused to obtain an intermediate fusion vector, until all the personality models are traversed; If all the personality models are traversed, the target question-answering model in the specified scenario is obtained based on the intermediate fusion vector.

8. A scenario question-answering model generation device, the device being used to implement the method according to any one of claims 1 to 7, characterized in that: The device comprises: an acquisition module, configured to acquire a plurality of sample question-answer pairs corresponding to each preset personality type, wherein the sample question-answer pairs include a sample question, a first sample answer, and a second sample answer, wherein the first sample answer matches the preset personality type, and the second sample answer does not match the preset personality type; A training module, for training a basic model using the plurality of sample question-answer pairs for each of the preset personality types, to obtain a personality model corresponding to the preset personality type, wherein an output distribution of the personality model is aligned with the first sample answer; A scenario processing module, used to generate a scenario answer corresponding to a scenario question using the personality model, and obtain a preference score of the personality model based on the scenario answer, wherein the preference score is positively correlated with human preference, and the scenario question is a question under a specified scenario; A fusion module, configured to fuse all personality models into a target question-answering model in the specified scenario based on the preference score of each personality model; The step of obtaining a plurality of sample question-answer pairs corresponding to each preset personality type includes: A basic dialogue data set is obtained, wherein the basic dialogue data set includes a plurality of sample questions; for each of the sample questions, a personality template corresponding to each preset personality type is generated based on a specified prompt word template, wherein the personality template includes a personality description corresponding to the preset personality type; the sample question is respectively combined with each of the personality templates to obtain a plurality of sample inputs corresponding to the sample question; for each of the sample inputs, the sample input is inferred through the basic model to obtain a sample reply matching each of the preset personality types; for each of the sample replies, an output value corresponding to the sample reply is calculated; with the output value corresponding to each sample reply, an output difference value of each sample pair is calculated, wherein the sample pair includes a first sample reply and a second sample reply that meet a preset condition, and the output difference value is the difference between the output value of the first sample reply and the output value of the second sample reply; if the output difference value is greater than a preset threshold value, the first sample reply, the second sample reply and the sample question are combined as a sample question-answer pair of the preset personality type corresponding to the first sample reply and the second sample reply.

Images