Model dialogue method and device based on style label, equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides a style label-based model dialogue method, device and equipment and a computer readable storage medium, and the method comprises the steps: integrating dialogue roles of various styles in a target dialogue model in advance, and then carrying out the multi-modal emotion recognition based on a facial image and the current dialogue content. And obtaining user emotion information. According to the dialogue role style label of each dialogue role in the target dialogue model or the dialogue style label corresponding to each piece of dialogue reply information generated by each dialogue role, target reply information is determined, so that the target dialogue model can switch different styles of dialogue reply information according to the current user dialogue state of the user; therefore, the intelligent degree of the intelligent dialogue and the matching degree of the user to the dialogue style are further improved, and the accuracy of the dialogue reply information is further improved.

Description

Technical Field

[0001] The present invention relates to the fields of natural language technology and artificial intelligence technology, and in particular to a model dialogue method, device, computer equipment and computer-readable storage medium based on style tags. Background Art

[0002] With the rapid development of society, people's pace of life is getting faster and faster, and the pressure of life is also increasing. With the popularization of large language model applications, large language models have also brought new application scenarios to many technologies, and have produced new product forms, one of which is the companionship intelligent dialogue system. The companionship intelligent dialogue system is an artificial intelligence-driven interactive system that can provide companionship and communication to users by simulating human dialogue partners, thereby reducing user pressure. The intelligent dialogue system can listen to the user's voice input, understand its meaning and generate a natural and fluent response, and then reply through artificial intelligence synthesized voice and video, creating an experience similar to talking to a real person. However, the current intelligent dialogue system has a relatively fixed dialogue style and cannot change the style according to the different dialogue states of the user, which reduces the user experience.

[0003] Therefore, how to change the conversation style according to the user's different conversation states to improve the user experience has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the present invention is to provide a model dialogue method, device, equipment and computer-readable storage medium based on style labels, aiming to change the dialogue style according to the different dialogue states of the user to improve the user experience.

[0005] To achieve the above object, the present invention provides a model dialogue method based on style tags, wherein the model dialogue comprises the following steps:

[0006] Based on the facial image of the current user and the current conversation content, multimodal emotion recognition is performed on the current user to obtain user emotion information;

[0007] Obtaining a dialogue role style label corresponding to each dialogue role in the target dialogue model, or generating each dialogue reply information corresponding to the user emotion information based on each dialogue role;

[0008] Based on the dialogue style tags corresponding to each dialogue reply message or the dialogue role style tags, the target reply message corresponding to the user emotion information is determined.

[0009] In addition, to achieve the above-mentioned purpose, the present invention further provides a model dialogue device based on style tags, and the model dialogue device based on style tags includes:

[0010] An emotion information generation module, used to perform multimodal emotion recognition on the current user based on the facial image of the current user and the current conversation content, to obtain user emotion information;

[0011] A dialogue role acquisition module, used to acquire dialogue role style labels corresponding to each dialogue role in the target dialogue model, or to generate dialogue reply information corresponding to the user emotion information based on each dialogue role;

[0012] The dialogue style reply module is used to determine the target reply information corresponding to the user emotion information based on the dialogue style tags corresponding to each dialogue reply information or the style tags of each dialogue role.

[0013] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a processor, a memory, and a style tag-based model dialogue program stored in the memory and executable by the processor, wherein when the style tag-based model dialogue program is executed by the processor, the steps of the style tag-based model dialogue method as described above are implemented.

[0014] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a style tag-based model dialogue program is stored, wherein when the style tag-based model dialogue program is executed by a processor, the steps of the style tag-based model dialogue method as described above are implemented.

[0015] The present application provides a model dialogue method based on style labels, the method performs multimodal emotion recognition on the current user based on the facial image of the current user and the current dialogue content to obtain user emotion information; obtains dialogue role style labels corresponding to each dialogue role in the target dialogue model, or generates each dialogue reply information corresponding to the user emotion information based on each dialogue role; determines the target reply information corresponding to the user emotion information based on the dialogue style labels corresponding to each dialogue reply information or each dialogue role style label. Through the above method, the present application pre-integrates dialogue roles of various styles in the target dialogue model, and then performs multimodal emotion recognition based on the facial image and the current dialogue content to obtain user emotion information. Then, according to the dialogue role style labels of each dialogue role in the target dialogue model or the dialogue style labels corresponding to each dialogue reply information generated by each dialogue role, the target reply information is determined, so that the target dialogue model can switch dialogue reply information of different styles according to the current user dialogue state of the user, thereby further improving the intelligence of the intelligent dialogue and the matching degree of the user to the dialogue style, thereby further improving the accuracy of the dialogue reply information. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the first embodiment of the model dialogue method based on style tags of the present invention;

[0017] Figure 2 It is a flow chart of a second embodiment of the model dialogue method based on style tags of the present invention;

[0018] Figure 3 It is a flow chart of a third embodiment of the model dialogue method based on style tags of the present invention;

[0019] Figure 4 A schematic diagram of a model dialogue device based on style tags of the present invention;

[0020] Figure 5 The figure is a schematic diagram of the hardware structure of a computer device involved in an embodiment of the present invention.

[0021] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0024] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0025] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0026] The style tag-based model dialogue method involved in the embodiment of the present invention is mainly applied to a computer device, which may be a device with display and processing functions such as a PC, a portable computer, or a mobile terminal.

[0027] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0028] Reference Figure 1 , Figure 1 It is a flowchart diagram of the first embodiment of the model dialogue method based on style tags of the present invention.

[0029] like Figure 1 As shown, an embodiment of the present invention provides a model dialogue method based on style labels, which can be applied to a companion intelligent dialogue system. The model dialogue method based on style labels includes steps S10 to S30.

[0030] Step S10: Based on the facial image of the current user and the current conversation content, multimodal emotion recognition is performed on the current user to obtain user emotion information;

[0031] In order to improve the user experience, in this embodiment, a conversation style suitable for the current user is determined based on the user's emotional state during the conversation.

[0032] Specifically, the facial expression of the current user is first obtained from the facial image of the current user, and the intonation information, semantic information and the text of the conversation content are obtained from the current conversation content sent by the current user to the target conversation model, so as to fuse the features of different modalities through a deep learning model based on the information of multiple modalities such as facial expression, intonation information, semantic information and text of the conversation content, and thus improve the accuracy of user emotion recognition. For example, using the Multimodal Emotion Recognition dataset, facial expressions, voice and text are combined to build a more comprehensive emotion recognition system.

[0033] Step S20: obtaining a dialogue role style label corresponding to each dialogue role in the target dialogue model, or generating each dialogue reply information corresponding to the user emotion information based on each dialogue role;

[0034] In this embodiment, multiple dialogue roles corresponding to different styles are pre-set, and a target dialogue model is constructed according to the style of each dialogue role. For example, a target dialogue model is constructed for the role of "female, gentle, and cafe owner", so that the language style of the content replied by the model during the dialogue with the user conforms to this role setting; target dialogue models can also be constructed for different dialogue role styles such as "male, confident, middle-level Internet leader", "genderless, sharp, and little devil", so as to obtain a target dialogue model containing multiple dialogue roles.

[0035] Specifically, each dialogue role can be described and determined using a label list consisting of multiple style labels. For example, the style label list for the role of "female, gentle, cafe owner" is: female, gentle, cafe owner, empathetic, positive, enthusiastic, etc.

[0036] A dialogue role style tag corresponding to the user emotion information may be searched, and a target dialogue role may be determined based on the corresponding dialogue role style tag, and then target response information may be generated based on the target dialogue role;

[0037] It is also possible to generate dialogue reply messages corresponding to the current dialogue content through multiple dialogue roles, and then determine the target reply message in each dialogue reply message according to the matching degree.

[0038] Furthermore, before obtaining the dialogue role style labels corresponding to the respective dialogue roles in the target dialogue model, or generating the respective dialogue reply information corresponding to the user emotion information based on the respective dialogue roles, the method further includes:

[0039] Based on the base dialogue model and the LORA parameter fine-tuning method, we train and generate various dialogue characters with different dialogue styles.

[0040] Specifically, prompt words are constructed in advance based on a list of style labels of determined dialogue roles, the dialogue history between the user and the dialogue model, and the user's dialogue content are input into the dialogue model for training to obtain each dialogue role in the target dialogue model, so that the dialogue model tends to generate dialogue response text corresponding to the relevant style labels.

[0041] The dataset for training the dialogue model is a mixture of dialogue data of various dialogue characters with different styles. In the process of model training, in addition to the dialogue history and the current user dialogue content, the style label list corresponding to the dialogue characters in the training data is also used to construct prompt words and input them into the model. The LORA fine-tuning method is applied to the base dialogue model to train a dialogue model that can generate different dialogue style requirements. By integrating the label information of the dialogue characters during the model training process, the consistency of the dialogue style generated by the model is enhanced.

[0042] When training a model using the LoRA (Low-Rank Adaptation) parameter fine-tuning method, most of the parameter weights of the base large model are first frozen, and then the model parameter weights are fine-tuned on a relatively small number of trainable low-rank matrices.

[0043] Step S30: determining target reply information corresponding to the user emotion information based on the dialogue style tags corresponding to each dialogue reply information or the dialogue role style tags.

[0044] In this embodiment, the dialogue role style label corresponding to the user emotion information may be searched, and based on the corresponding dialogue role style label, the target dialogue role may be determined, and then the target reply information may be generated based on the target dialogue role;

[0045] Alternatively, multiple dialogue roles are used to generate dialogue reply messages corresponding to the current dialogue content, and then the reply message with the highest matching degree is determined among the dialogue reply messages as the target reply message.

[0046] This embodiment provides a model dialogue method based on style labels, the method performs multimodal emotion recognition on the current user based on the facial image of the current user and the current dialogue content to obtain user emotion information; obtains dialogue role style labels corresponding to each dialogue role in the target dialogue model, or generates each dialogue reply information corresponding to the user emotion information based on each dialogue role; determines the target reply information corresponding to the user emotion information based on the dialogue style labels corresponding to each dialogue reply information or each dialogue role style label. In the above manner, the present application pre-integrates dialogue roles of various styles in the target dialogue model, and then performs multimodal emotion recognition based on the facial image and the current dialogue content to obtain user emotion information. Then, according to the dialogue role style labels of each dialogue role in the target dialogue model or the dialogue style labels corresponding to each dialogue reply information generated by each dialogue role, the target reply information is determined, so that the target dialogue model can switch dialogue reply information of different styles according to the current user dialogue state of the user, thereby further improving the intelligence of the intelligent dialogue and the matching degree of the user to the dialogue style, thereby further improving the accuracy of the dialogue reply information.

[0047] Reference Figure 2 , Figure 2 Schematic diagram of the flow chart of the second embodiment of the model dialogue method based on style tags of the present invention.

[0048] Based on the above Figure 1 In the embodiment shown, in this embodiment, the step S30 specifically includes:

[0049] Step S31, determining the matching degree between each dialogue reply message and the user emotion information based on the dialogue style matching model and the dialogue style label corresponding to each dialogue reply message;

[0050] Step S32, determining the dialogue reply information with the highest matching degree among the dialogue reply information as the target reply information.

[0051] In this embodiment, N different dialogue replies are generated by each dialogue role in the target dialogue model. N can be set according to the number of dialogue roles in the model. For example, N can be 8, that is, 8 different dialogue replies are generated at a time. Among these N dialogue replies, the one with the highest style matching degree and the set dialogue role style is selected as the final dialogue reply of the model.

[0052] By establishing a dialogue style matching model, the style matching degree of each dialogue content is evaluated. The input of the model is a dialogue reply content and a set dialogue role style label. The output of the model is between 0 and 1. The larger the number, the higher the matching degree. The training data of the model is constructed. For each non-user reply content (i.e., the reply content of the dialogue role) in the collected different dialogue data, the list of dialogue role style labels that are consistent with it is set to 1 to indicate a match, and multiple other inconsistent dialogue style labels randomly selected from the dialogue role style labels are set to 0 to indicate a mismatch. The details are shown in the following table:

[0053]

[0054] Furthermore, the step S31 includes:

[0055] A twin network structure is constructed based on the text pre-trained BERT network as the network structure of the conversation style matching model;

[0056] Encoding the dialogue style label corresponding to each dialogue reply message and the user emotion information based on the BERT network to obtain a feature vector corresponding to each dialogue reply message and a feature vector corresponding to the user emotion information;

[0057] Concatenating the feature vector corresponding to the user emotion information with the feature vector corresponding to each dialogue reply information;

[0058] Each concatenated vector is input into the fully connected network and the classification network of the dialogue style matching model to obtain the matching degree between each dialogue reply information and the user emotion information.

[0059] In this embodiment, the network structure of the dialogue style matching model is a twin network structure (Siamese Network) constructed based on the text pre-trained BERT network. The text is converted into a corresponding text feature vector through the same sentence BERT network as an encoder, and then each text feature vector is matched to complete the style matching evaluation task. Specifically, the reply content of the dialogue character and the character style label list are encoded through the BERT network to obtain the text feature vector v and the character style feature vector u. Select a dialogue character in turn, splice the two feature vectors v and u corresponding to the dialogue character and the user's emotional information, respectively, connect to the fully connected network, output 0 or 1 through softmax classification, use cross entropy as the loss function, and stop training the model when the loss function reaches the preset value to obtain a dialogue style matching model.

[0060] Through the above method, this embodiment first establishes a label system for different dialogue roles, and in the process of training the dialogue model, the data of multiple different dialogue roles are mixed to fuse the label information of the set dialogue role, so that the dialogue model can generate reply content of specific styles for different dialogue roles; in addition, a dialogue style evaluation mechanism is established. When the dialogue model generates dialogue reply information, the mechanism is applied to call the dialogue style matching model to check the consistency of the style of the dialogue content, and determine the reply with the highest style consistency as the final dialogue output. In this way, a dialogue model for multiple dialogue role styles is realized, and the user's experience of healing dialogue is improved by improving the personalization and style consistency of the generated dialogue.

[0061] Reference Figure 3 , Figure 3 It is a flowchart diagram of the third embodiment of the model dialogue method based on style tags of the present invention.

[0062] Based on the above Figure 1 In the embodiment shown, in this embodiment, the step S30 specifically includes:

[0063] Step S33, determining a target dialogue role that matches the user emotion information based on the dialogue style matching model and the style labels of each dialogue role;

[0064] Step S34: generating target reply information corresponding to the user emotion information based on the target dialogue role.

[0065] In this embodiment, the dialogue style matching model can be trained in advance, and dialogue role style labels corresponding to various emotional information can be established through the dialogue style matching model. For example, the emotion of loss corresponds to the role of "female, gentle, cafe owner", and the emotion of lack of confidence and prone to anxiety corresponds to the role of "male, confident, middle-level Internet leader", etc.

[0066] In each of the dialogue role style tags, the dialogue role style tag corresponding to the user emotion information is searched, and based on the corresponding dialogue role style tag, the target dialogue role is determined, and then the target reply information is generated based on the target dialogue role.

[0067] Furthermore, before step S31, the following steps are also included:

[0068] Initialize the weight parameters and bias parameters of each network layer in the initial model, and generate the survival probability parameters of each network layer based on preset rules;

[0069] Based on the survival probability parameters of each network layer, a target network layer and an optimized network layer are determined in each network layer, wherein the target network layer is a network layer to be trained, and the optimized network layer is a network layer for which training needs to be skipped;

[0070] Based on a preset dialogue label sample set, initial weight parameters, and initial bias parameters, the optimization network layer is skipped, and the target network layer is trained until a preset convergence condition is reached, thereby obtaining a trained model as the dialogue style matching model.

[0071] Exemplarily, the determining of the target network layer and optimizing the network layer at each network layer based on the survival probability parameters of each network layer specifically includes:

[0072] Obtain any value within a preset value range as a random parameter of a network layer;

[0073] Compare the survival probability parameter of each network layer with its corresponding random parameter;

[0074] Based on the comparison results of each network layer, the network layer whose survival probability parameter is not less than the random parameter is determined as the target network layer, and the network layer whose survival probability parameter is less than the random parameter is determined as the optimized network layer.

[0075] In this embodiment, in order to prevent the dialogue style matching model from overfitting or underfitting during training, thereby causing the model training time to be too long, a random depth mechanism is provided based on the survival probability corresponding to each network layer in the model. Based on the random depth mechanism, some network layers can be randomly skipped during the training process, reducing the amount of forward and backward propagation calculations during model training, thereby reducing the computational complexity during model training; and only some network layers are used in each training, so that the training time is significantly shortened while ensuring the convergence of the model, preventing the model from overfitting, and enhancing the model's robustness to abnormal data and noise data. In addition, the label generation model obtained by training based on the random depth mechanism can further improve the model training efficiency and label generation accuracy while ensuring model performance.

[0076] Specifically, the network parameters and the survival probability of each layer are initialized first. Then, in each forward propagation process, it is randomly decided whether to skip the layer according to the survival probability of each layer. For the network layers participating in the training, the parameters are updated by forward and backward propagation as usual. For the network layers not participating in the training, the parameters are not updated, thereby realizing multiple iterative training of the model to ensure that the model converges and achieves the optimal performance.

[0077] Exemplarily, the step S10 specifically includes:

[0078] Initializing the weight parameters and bias parameters of each network layer based on the He initialization method or the Xavier initialization method to obtain the initial weight parameters and the initial bias parameters;

[0079] Based on a linear decreasing strategy, the survival probability parameters of each network layer are generated, wherein the level of the network layer is inversely proportional to the corresponding survival probability parameter value.

[0080] In this embodiment, by introducing a random depth mechanism in the training process, efficient training of a deep neural network is achieved, specifically including:

[0081] 1. Initialize the network parameters, that is, initialize the weight parameters and bias parameters in each layer of the random deep neural network. Specifically, the model parameters of the random deep neural network can be initialized by using methods such as He initialization or Xavier initialization to ensure the stability and effectiveness of the training.

[0082] Second, initialize the survival probability, that is, assign an initial survival probability p to each layer of the network l .

[0083] Specifically, a linear decreasing strategy can be adopted, that is, the level of the network layer is inversely proportional to the corresponding survival probability parameter value. By adopting a linear decreasing survival probability design, the effective depth of the network during the training process is guaranteed, while avoiding overly complex calculations.

[0084] That is, the survival probability is higher near the input layer, while the survival probability is lower near the output layer. For example, the survival probability can be set as:

[0085]

[0086] Among them, l is the current network layer, L is the total number of network layers, and p L It is the survival probability of the output layer in the previous round. The initial value can be taken in the first round. The value is generally low, such as less than 50%.

[0087] Specifically, a preset probability threshold can be set. When the survival probability parameter is not less than the preset probability threshold, the network layer can be recorded as the target network layer to train the network layer. When the survival probability parameter is less than the preset probability threshold, the network layer can be recorded as the optimized network layer to skip the network layer.

[0088] The initial model is a random deep network model with initial weight parameters and initial bias parameters as the current model parameters. In each forward propagation process, the basic information of the contacts in the preset conversation label sample set is input into each target network layer in the initial random deep network model, that is, the output results of each target network layer skip the next layer of the optimization network layer and directly pass it to the next layer of the target network layer. After completing the training of all target network layers in the random deep network model, it is determined whether the current random deep network model has reached the convergence condition. If not, the target network layer in the random deep network model is iteratively trained again, and the optimization network layer is still skipped until the random deep network model reaches the preset convergence condition. Stop model training and obtain the trained preset conversation label sample set.

[0089] Furthermore, the determining of the target network layer and optimizing the network layer at each network layer based on the survival probability parameters of each network layer specifically includes:

[0090] Obtain any value within a preset value range as a random parameter of a network layer;

[0091] Compare the survival probability parameter of each network layer with its corresponding random parameter;

[0092] Based on the comparison results of each network layer, the network layer whose survival probability parameter is not less than the random parameter is determined as the target network layer, and the network layer whose survival probability parameter is less than the random parameter is determined as the optimized network layer.

[0093] First, the emotion samples in the preset dialogue label sample set are input into the first-level network layer.

[0094] Then, each network layer except the first-level network layer in the random deep network model is processed layer by layer, and each network layer of the initialized random deep network model is forward propagated:

[0095] For each network layer l, generate a random number r between [0,1] l , as random parameters of each level of network layer;

[0096] The random parameters r of each network layer are l The corresponding survival probability parameter p l To compare:

[0097] 1. If r l≤p l , then this layer participates in the calculation and performs normal forward propagation:

[0098] x l+1 =f l (x l ,W l ,b l )

[0099] Among them, x l is the input of the lth layer, W l and b l are the weight and bias of the lth layer, respectively, and f l is the activation function of this layer.

[0100] 2. If r l >p l , then skip this layer and pass the current input directly to the next layer:

[0101] x l+1 =x l

[0102] In the above manner, this embodiment generates corresponding random parameters for each level of network layer to ensure the randomness of skipping the layer, and then compares the random parameters of the layer with its corresponding survival probability parameters, and determines whether to skip the layer during training based on the comparison results. Thus, the randomness of the optimized network layer is ensured based on the random parameters, thereby avoiding the fixed probability parameter threshold leading to a fixed optimized network layer, and further improving the accuracy of the prediction results of the random deep network model.

[0103] After completing a round of forward propagation of the model, back propagation is performed to update the parameters of the network layers involved in the forward propagation. Specifically:

[0104] 1. Calculate the label sample corresponding to the dialogue character style label sample (i.e. the target value y true ) and the loss function of the current dialogue character style label (network output y):

[0105] According to the network output y and target value y true Calculate the loss function L(y,y true ), using cross entropy loss or mean squared error loss.

[0106] 2. Back propagation layer by layer:

[0107] Starting from the output layer, the gradient is calculated forward layer by layer.

[0108] For each layer l, if the layer participates in the forward propagation (i.e., r l ≤p l ), the gradient is calculated normally and the parameters are updated:

[0109]

[0110] Among them, η is the preset learning rate.

[0111] If the layer is skipped (i.e. r l >p l ), no parameter update is performed and the gradient is directly passed to the previous layer:

[0112]

[0113] In the above manner, this embodiment will iterate multiple times on the entire training set based on the above forward propagation and back propagation steps until the loss function converges or reaches a preset number of training rounds. During each iteration, the survival probability p of each network layer in this round is l Keep it unchanged to ensure the stability and effect of model training.

[0114] See also Figure 4 , Figure 4 It is a schematic diagram of functional modules of a model dialogue device based on style tags provided in an embodiment of the present application.

[0115] like Figure 4 As shown, the style tag-based model dialogue device 400 includes:

[0116] The emotion information generating module 410 is used to perform multimodal emotion recognition on the current user based on the facial image of the current user and the current conversation content to obtain the user emotion information;

[0117] A dialogue role acquisition module 420 is used to acquire a dialogue role style label corresponding to each dialogue role in the target dialogue model, or to generate each dialogue reply information corresponding to the user emotion information based on each dialogue role;

[0118] The dialogue style reply module 430 is used to determine the target reply information corresponding to the user emotion information based on the dialogue style tags corresponding to each dialogue reply information or the style tags of each dialogue role.

[0119] Furthermore, the conversation style reply module 430 specifically includes:

[0120] A matching degree determining unit, configured to determine a matching degree between each dialogue reply message and the user emotion information based on a dialogue style matching model and a dialogue style label corresponding to each dialogue reply message;

[0121] The reply information determination unit is used to determine the dialogue reply information with the highest matching degree among the dialogue reply information as the target reply information.

[0122] Furthermore, the dialogue style reply module 430 also includes a matching model training module, and the matching model training module specifically includes:

[0123] A probability parameter generating unit, used to initialize the weight parameters and bias parameters of each network layer in the initial model, and generate the survival probability parameters of each network layer based on preset rules;

[0124] A network layer classification unit, used to determine a target network layer and an optimized network layer in each network layer based on the survival probability parameters of each network layer, wherein the target network layer is a network layer to be trained, and the optimized network layer is a network layer for which training needs to be skipped;

[0125] The matching model generation unit is used to skip the optimization network layer based on a preset dialogue tag sample set, an initial weight parameter, and an initial bias parameter, and train the target network layer until a preset convergence condition is reached to obtain a trained model as the dialogue style matching model.

[0126] Furthermore, the network layer classification unit specifically includes:

[0127] A random parameter generation subunit, used to obtain any value within a preset value range as a random parameter of a network layer;

[0128] A network layer parameter comparison unit, used to compare the survival probability parameter of each network layer with its corresponding random parameter;

[0129] The network layer classification subunit is used to determine, based on the comparison results of each network layer, the network layer whose survival probability parameter is not less than the random parameter as the target network layer, and determine the network layer whose survival probability parameter is less than the random parameter as the optimized network layer.

[0130] Furthermore, the matching degree determination unit includes:

[0131] A network structure determination subunit, used to construct a twin network structure based on a text pre-trained BERT network as the network structure of the conversation style matching model;

[0132] A feature vector encoding subunit, used to encode the dialogue style label corresponding to each dialogue reply information and the user emotion information based on the BERT network to obtain a feature vector corresponding to each dialogue reply information and a feature vector corresponding to the user emotion information;

[0133] A feature vector concatenation subunit, used to concatenate the feature vector corresponding to the user emotion information with the feature vector corresponding to each dialogue reply information;

[0134] The matching degree determination subunit is used to input each concatenated vector into the fully connected network and the classification network of the dialogue style matching model to obtain the matching degree of each dialogue reply information and the user emotion information.

[0135] Furthermore, the style tag-based model dialogue device 400 further includes:

[0136] The dialogue role training module is used to train and generate various dialogue roles with different dialogue styles based on the base dialogue model and the LORA parameter fine-tuning method.

[0137] Furthermore, the conversation style reply module 430 further includes:

[0138] A dialogue role determination unit, configured to determine a target dialogue role that matches the user emotion information based on a dialogue style matching model and each of the dialogue role style tags;

[0139] A reply information generating unit is used to generate target reply information corresponding to the user emotion information based on the target dialogue role.

[0140] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0141] The above-mentioned device can be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.

[0142] See also Figure 5 , Figure 5 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0143] See also Figure 5 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0144] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any style tag-based model dialogue method.

[0145] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0146] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any style tag-based model dialogue method.

[0147] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0148] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0149] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0150] Based on the facial image of the current user and the current conversation content, multimodal emotion recognition is performed on the current user to obtain user emotion information;

[0151] Obtaining a dialogue role style label corresponding to each dialogue role in the target dialogue model, or generating each dialogue reply information corresponding to the user emotion information based on each dialogue role;

[0152] Based on the dialogue style tags corresponding to each dialogue reply message or the dialogue role style tags, the target reply message corresponding to the user emotion information is determined.

[0153] In one embodiment, the processor is used to process the dialogue style tags corresponding to each dialogue reply information or each dialogue role style tag, and determine the target reply information corresponding to the user emotion information, and is also used to implement:

[0154] Determining the matching degree between each dialogue reply message and the user emotion information based on the dialogue style matching model and the dialogue style label corresponding to each dialogue reply message;

[0155] The dialogue reply information with the highest matching degree is determined among the dialogue reply information as the target reply information.

[0156] In one embodiment, the processor is used to process the dialogue style matching model and each of the dialogue role style tags, and before determining the target dialogue role that matches the user emotion information, further used to implement:

[0157] Initialize the weight parameters and bias parameters of each network layer in the initial model, and generate the survival probability parameters of each network layer based on preset rules;

[0158] Based on the survival probability parameters of each network layer, a target network layer and an optimized network layer are determined in each network layer, wherein the target network layer is a network layer to be trained, and the optimized network layer is a network layer for which training needs to be skipped;

[0159] Based on a preset dialogue label sample set, initial weight parameters, and initial bias parameters, the optimization network layer is skipped, and the target network layer is trained until a preset convergence condition is reached, thereby obtaining a trained model as the dialogue style matching model.

[0160] In one embodiment, the processor is used to process the survival probability parameters based on each network layer, and when each network layer determines the target network layer and optimizes the network layer, it is also used to implement:

[0161] Obtain any value within a preset value range as a random parameter of a network layer;

[0162] Compare the survival probability parameter of each network layer with its corresponding random parameter;

[0163] Based on the comparison results of each network layer, the network layer whose survival probability parameter is not less than the random parameter is determined as the target network layer, and the network layer whose survival probability parameter is less than the random parameter is determined as the optimized network layer.

[0164] In one embodiment, the processor is used to process the dialogue style tag corresponding to each dialogue reply information based on the dialogue style matching model and determine the matching degree between each dialogue reply information and the user emotion information, and is also used to implement:

[0165] A twin network structure is constructed based on the text pre-trained BERT network as the network structure of the conversation style matching model;

[0166] Encoding the dialogue style label corresponding to each dialogue reply message and the user emotion information based on the BERT network to obtain a feature vector corresponding to each dialogue reply message and a feature vector corresponding to the user emotion information;

[0167] Concatenating the feature vector corresponding to the user emotion information with the feature vector corresponding to each dialogue reply information;

[0168] Each concatenated vector is input into the fully connected network and the classification network of the dialogue style matching model to obtain the matching degree between each dialogue reply information and the user emotion information.

[0169] In one embodiment, the processor is used to process the acquisition of the dialogue role style tags corresponding to each dialogue role in the target dialogue model, or before generating each dialogue reply information corresponding to the user emotion information based on each dialogue role, further used to implement:

[0170] Based on the base dialogue model and the LORA parameter fine-tuning method, we train and generate various dialogue characters with different dialogue styles.

[0171] In one embodiment, the processor is used to process the dialogue style tags corresponding to each dialogue reply information or each dialogue role style tag, and determine the target reply information corresponding to the user emotion information, and is also used to implement:

[0172] Determining a target dialogue role that matches the user emotion information based on a dialogue style matching model and each of the dialogue role style labels;

[0173] Based on the target dialogue role, target reply information corresponding to the user emotion information is generated.

[0174] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any style tag-based model dialogue method provided in the embodiment of the present application.

[0175] The computer-readable storage medium may be an internal storage unit of the computer device described in the above embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.

[0176] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A model dialogue method based on style tags, characterized in that: The model dialogue includes the following steps: Based on the facial image of the current user and the current conversation content, multimodal emotion recognition is performed on the current user to obtain user emotion information; Obtaining a dialogue role style label corresponding to each dialogue role in the target dialogue model, or generating each dialogue reply information corresponding to the user emotion information based on each dialogue role; Based on the dialogue style tags corresponding to each dialogue reply message or the dialogue role style tags, the target reply message corresponding to the user emotion information is determined.

2. The model dialogue method based on style tags as claimed in claim 1, characterized in that: The determining, based on the dialogue style tags corresponding to each dialogue reply message or each dialogue role style tag, the target reply message corresponding to the user emotion information specifically includes: Determining the matching degree between each dialogue reply message and the user emotion information based on the dialogue style matching model and the dialogue style label corresponding to each dialogue reply message; The dialogue reply information with the highest matching degree is determined among the dialogue reply information as the target reply information.

3. The model dialogue method based on style tags as claimed in claim 2, characterized in that: The determining, based on the dialogue style matching model and the dialogue style label corresponding to each dialogue reply message, the matching degree between each dialogue reply message and the user emotion information comprises: A twin network structure is constructed based on the text pre-trained BERT network as the network structure of the conversation style matching model; Encoding the dialogue style label corresponding to each dialogue reply message and the user emotion information based on the BERT network to obtain a feature vector corresponding to each dialogue reply message and a feature vector corresponding to the user emotion information; Concatenating the feature vector corresponding to the user emotion information with the feature vector corresponding to each dialogue reply information; Each concatenated vector is input into the fully connected network and the classification network of the dialogue style matching model to obtain the matching degree between each dialogue reply information and the user emotion information.

4. The model dialogue method based on style tags as claimed in claim 1, characterized in that: The determining, based on the dialogue style tags corresponding to each dialogue reply message or each dialogue role style tag, the target reply message corresponding to the user emotion information further includes: Determining a target dialogue role that matches the user emotion information based on a dialogue style matching model and each of the dialogue role style labels; Based on the target dialogue role, target reply information corresponding to the user emotion information is generated.

5. The model dialogue method based on style tags as claimed in claim 4, characterized in that: Before determining the target dialogue role that matches the user emotion information based on the dialogue style matching model and each of the dialogue role style tags, the method further includes: Initialize the weight parameters and bias parameters of each network layer in the initial model, and generate the survival probability parameters of each network layer based on preset rules; Based on the survival probability parameters of each network layer, a target network layer and an optimized network layer are determined in each network layer, wherein the target network layer is a network layer to be trained, and the optimized network layer is a network layer for which training needs to be skipped; Based on a preset dialogue label sample set, initial weight parameters, and initial bias parameters, the optimization network layer is skipped, and the target network layer is trained until a preset convergence condition is reached, thereby obtaining a trained model as the dialogue style matching model.

6. The model dialogue method based on style tags as claimed in claim 5, characterized in that: The determining of the target network layer and optimizing the network layer at each network layer based on the survival probability parameters of each network layer specifically includes: Obtain any value within a preset value range as a random parameter of a network layer; Compare the survival probability parameter of each network layer with its corresponding random parameter; Based on the comparison results of each network layer, the network layer whose survival probability parameter is not less than the random parameter is determined as the target network layer, and the network layer whose survival probability parameter is less than the random parameter is determined as the optimized network layer.

7. The model dialogue method based on style tags according to any one of claims 1 to 6, characterized in that: Before acquiring the dialogue role style labels corresponding to the respective dialogue roles in the target dialogue model, or generating the respective dialogue reply information corresponding to the user emotion information based on the respective dialogue roles, the method further includes: Based on the base dialogue model and the LORA parameter fine-tuning method, we train and generate various dialogue characters with different dialogue styles.

8. A model dialogue device based on style tags, characterized in that: The model dialogue device based on style tags includes: An emotion information generation module, used to perform multimodal emotion recognition on the current user based on the facial image of the current user and the current conversation content, to obtain user emotion information; A dialogue role acquisition module, used to acquire dialogue role style labels corresponding to each dialogue role in the target dialogue model, or to generate dialogue reply information corresponding to the user emotion information based on each dialogue role; The dialogue style reply module is used to determine the target reply information corresponding to the user emotion information based on the dialogue style tags corresponding to each dialogue reply information or the style tags of each dialogue role.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a style tag-based model dialogue program stored in the memory and executable by the processor, wherein when the style tag-based model dialogue program is executed by the processor, the steps of the style tag-based model dialogue method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a style tag-based model dialogue program, wherein when the style tag-based model dialogue program is executed by a processor, the steps of the style tag-based model dialogue method according to any one of claims 1 to 7 are implemented.

Images