Method, device and electronic device for determining speech strategy for voice conversation

By analyzing user data from the perspective of speech and semantics, using deep learning models to generate emotional and attitude labels, and determining speech strategies, the problem of intelligent voice robots not responding well in human-computer dialogues is solved, and dialogue quality and user satisfaction are improved.

CN113935336BActive Publication Date: 2024-11-19SHANGHAI QIYUE INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111175425.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-09
Publication Date
2024-11-19
Estimated Expiration
2041-10-09

AI Technical Summary

Technical Problem

Intelligent voice robots find it difficult to quickly and accurately reflect user emotions and attitudes in human-computer dialogues, affecting conversation quality and user satisfaction.

Method used

By analyzing the user's real-time speech data from the perspectives of speech and semantics, extracting speech features and semantic features, using deep learning models (such as long and short-term memory networks and recurrent neural networks) to generate emotional labels and attitude labels, determine speech strategies, and then Optimize reply data to improve conversation quality.

Benefits of technology

The rapid accuracy of voice conversations and improved user satisfaction are achieved, and the real human response can be better simulated.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113935336B_ABST
    Figure CN113935336B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a method, device, electronic device and computer-readable medium for determining a speech strategy for voice conversations. The method comprises: obtaining real-time voice data of a user during a voice conversation with the user; extracting voice feature data of the real-time voice data based on a voice model to determine the emotion tag of the user; extracting semantic feature data of the real-time voice data based on a semantic model to determine the attitude tag of the user; determining a speech strategy based on the emotion tag and the attitude tag; and determining reply data based on the speech strategy to continue the voice conversation with the user. The present disclosure analyzes the user's voice data from the perspectives of voice and semantics, respectively, and can make full use of the information in the voice data to quickly and accurately determine the speech tag, thereby improving the quality and accuracy of the voice conversation and improving user satisfaction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer information processing, and in particular, to a method, device, electronic device and computer-readable medium for determining a speech strategy for voice conversation. Background Art

[0002] Based on technologies such as speech recognition and synthesis, machine learning, and natural language understanding, intelligent voice robots automatically initiate outbound calls based on business scenarios, collect business results through voice dialogues between people and robots, and statistically process data to obtain user feedback. Intelligent voice robots are conversational intelligent robots for developers, which can implement intelligent conversations based on natural language processing (NLP) on different message terminals, such as websites, apps, and physical robots. Intelligent voice robots are used in various fields and can partially replace outbound calls by human agents, and can perform return visits, notifications, surveys, marketing, and other services. Companies can configure their own unique knowledge base to achieve intelligent question and answer, or they can integrate with third-party APIs through multiple rounds of dialogue to achieve self-service, such as order inquiries, logistics tracking, self-service return robots, etc.

[0003] Human-computer dialogue is a sub-direction in the field of artificial intelligence. In layman's terms, it allows people to interact with computers through human language (i.e. natural language). As one of the ultimate challenges of artificial intelligence, a complete human-computer dialogue system involves a wide range of technologies, such as speech technology in computer science, natural language processing, machine learning, planning and reasoning, knowledge engineering, and even many theories in linguistics and cognitive science are applied in human-computer dialogue. Generally speaking, human-computer dialogue can be divided into the following four sub-problems: natural language chat, task-driven multi-round dialogue, question and answer, and recommendation. The above four types of problems have different problems to solve and different implementation technologies, but they all face a unified technical problem, that is, how can intelligent robots respond quickly and accurately to user feedback so that human-computer dialogue can proceed smoothly and intelligent robots can simulate the dialogue and response of real people as much as possible.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0005] In view of this, the present disclosure provides a method, device, electronic device and computer-readable medium for determining speech strategies for voice conversations, which analyze the user's voice data from both phonetic and semantic perspectives, respectively, and can make full use of the information in the voice data to quickly and accurately determine speech tags, thereby improving the quality and accuracy of voice conversations and improving user satisfaction.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a method for determining a speech strategy for voice conversation is proposed, the method comprising: acquiring real-time voice data of the user during a voice conversation with the user; extracting voice feature data of the real-time voice data based on a voice model to determine an emotion label of the user; extracting semantic feature data of the real-time voice data based on a semantic model to determine an attitude label of the user; determining a speech strategy based on the emotion label and the attitude label; and determining reply data based on the speech strategy to continue the voice conversation with the user.

[0008] Optionally, the method further includes: acquiring historical voice data of historical users; determining emotion tags for the historical voice data; and training a long short-term memory network model based on the historical voice data with emotion tags to generate the voice model.

[0009] Optionally, the long short-term memory network model is trained based on historical speech data with emotion labels to generate the speech model, including: extracting historical speech feature data of the historical speech data through Mel-cephalogram coefficients; inputting the historical speech feature data with emotion labels into the long short-term memory network model for training; and generating the speech model when the activation function softmax function satisfies the training strategy.

[0010] Optionally, the method further includes: acquiring historical voice data of historical users; determining attitude tags for the historical voice data; and training a recurrent neural network model based on the historical voice data with attitude tags to generate the semantic model.

[0011] Optionally, a recurrent neural network model is trained based on historical speech data with attitude labels to generate the semantic model, including: converting the historical speech data into historical text data; generating historical text feature data through the historical text data; inputting the historical text feature data with attitude labels into the recurrent neural network model for training; and generating the semantic model when the activation function tanh function satisfies the training strategy.

[0012] Optionally, extracting speech feature data of the real-time speech data based on a speech model to determine the user's emotion label includes: performing data preprocessing on the real-time speech data to generate frame data; performing fast Fourier transform on the frame data to generate transformation data; filtering the transformation data through multiple Mel filter groups to generate speech feature data; and inputting the speech feature data into the speech model to generate the user's emotion label.

[0013] Optionally, extracting semantic feature data of the real-time voice data based on a semantic model to determine the attitude label of the user includes: converting the real-time voice data into text data; performing text processing on the text data to generate text feature data; and inputting the text feature data into the semantic model to generate the attitude label of the user.

[0014] Optionally, determining the speech strategy based on the emotion tag and the attitude tag includes: obtaining user data of the user, the user data including basic data and behavior data; determining a user portrait of the user based on the user data; and determining the speech strategy based on the user portrait and the emotion tag and the attitude tag.

[0015] Optionally, reply data is determined based on the speech strategy to continue the voice conversation with the user, including: determining reply voice data or reply text data based on the speech tag; and having a voice conversation with the user based on the reply voice data or reply text data.

[0016] According to one aspect of the present disclosure, a device for determining a speech strategy for voice conversation is proposed, the device comprising: an acquisition module, for acquiring real-time voice data of a user during a voice conversation with the user; a voice module, for extracting voice feature data of the real-time voice data based on a voice model to determine an emotion label of the user; a semantic module, for extracting semantic feature data of the real-time voice data based on a semantic model to determine an attitude label of the user; a strategy module, for determining a speech strategy based on the emotion label and the attitude label; and a reply module, for determining reply data based on the speech strategy to continue the voice conversation with the user.

[0017] According to one aspect of the present disclosure, an electronic device is proposed, which includes: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0018] According to one aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0019] According to the disclosed method, device, electronic device and computer-readable medium for determining speech strategy for voice conversation, in the process of having a voice conversation with the user, the user's real-time voice data is obtained; based on the voice model, the voice feature data of the real-time voice data is extracted to determine the user's emotion label; based on the semantic model, the semantic feature data of the real-time voice data is extracted to determine the user's attitude label; the speech strategy is determined based on the emotion label and the attitude label; based on the speech strategy, the mode of replying data to continue the voice conversation with the user is determined. The user's voice data is analyzed from both the voice and semantic perspectives, and the information in the voice data can be fully utilized to quickly and accurately determine the speech label, thereby improving the quality and accuracy of the voice conversation and improving user satisfaction.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail its exemplary embodiments with reference to the accompanying drawings. The accompanying drawings described below are only some embodiments of the present disclosure, and it is clear to a person skilled in the art that other accompanying drawings can be obtained from these accompanying drawings without creative work.

[0022] Figure 1 It is a system block diagram of a method and device for determining a speech strategy for voice conversation according to an exemplary embodiment.

[0023] Figure 2 The present invention is a flowchart of a method for determining a speech strategy for voice conversation according to an exemplary embodiment.

[0024] Figure 3 The present invention is a flowchart of a method for determining a speech strategy for voice conversation according to another exemplary embodiment.

[0025] Figure 4 The present invention is a flowchart of a method for determining a speech strategy for voice conversation according to another exemplary embodiment.

[0026] Figure 5 The present invention is a flowchart of a method for determining a speech strategy for voice conversation according to another exemplary embodiment.

[0027] Figure 6 It is a block diagram of a device for determining a speech strategy for voice conversation according to an exemplary embodiment.

[0028] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment.

[0029] Figure 8 It is a block diagram of a computer-readable medium according to an exemplary embodiment. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0031] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0032] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0033] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0034] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the disclosed concepts. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0035] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present disclosure, and therefore cannot be used to limit the protection scope of the present disclosure.

[0036] Figure 1 It is a system block diagram of a method and device for determining a speech strategy for voice conversation according to an exemplary embodiment.

[0037] like Figure 1 As shown, the system architecture 10 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0038] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as financial service applications, shopping applications, web browser applications, instant messaging tools, email clients, social platform software, etc.

[0039] The terminal devices 101, 102, 103 may be electronic devices with a call function, or various electronic devices supporting web browsing, including but not limited to ordinary telephones, smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0040] The server 105 may be a server that provides various services, for example, a server that provides speech strategy support for the user's voice service request. The speech strategy support server may analyze the received real-time voice data and feed back the processing results (e.g., reply data) to the voice dialogue personnel and / or the terminal devices 101, 102, 103. The server 105 may also be, for example, a backend server that provides support for the website browsed by the user using the terminal devices 101, 102, 103. The backend server may conduct a web-based voice call with the user based on the web page. The backend server may analyze the received real-time voice data and feed back the processing results (e.g., reply data) to the voice dialogue personnel and / or the terminal devices 101, 102, 103.

[0041] The server 105 may, for example, obtain real-time voice data of the user during a voice conversation with the user; the server 105 may, for example, extract voice feature data of the real-time voice data based on a voice model to determine the emotion tag of the user; the server 105 may, for example, extract semantic feature data of the real-time voice data based on a semantic model to determine the attitude tag of the user; the server 105 may, for example, determine a speech strategy based on the emotion tag and the attitude tag; the server 105 may, for example, determine reply data based on the speech strategy to continue the voice conversation with the user.

[0042] The server 105 may also, for example, obtain historical speech data of historical users; determine emotion tags for the historical speech data; and train a long short-term memory network model based on the historical speech data with emotion tags to generate the speech model.

[0043] The server 105 may also, for example, obtain historical speech data of historical users; determine attitude tags for the historical speech data; and train a recurrent neural network model based on the historical speech data with attitude tags to generate the semantic model.

[0044] Server 105 can be a physical server, or can be composed of multiple servers, for example. A part of server 105 can be used as the voice dialogue system in the present disclosure, for determining the reply data based on the speech strategy to continue the voice dialogue with the user; and a part of server 105 can also be used as the model training system in the present disclosure, for training the long short-term memory network model based on the historical speech data with emotion labels to generate the speech model, and / or training the recurrent neural network model based on the historical speech data with attitude labels to generate the semantic model.

[0045] It should be noted that the method for determining the speech strategy for voice conversation provided in the embodiment of the present disclosure can be executed by the server 105 , and accordingly, the device for determining the speech strategy for voice conversation can be set in the server 105 .

[0046] Figure 2 The flowchart of a method for determining speech strategy for voice dialogue according to an exemplary embodiment is shown. The method 20 for determining speech strategy for voice dialogue at least includes steps S202 to S210.

[0047] like Figure 2 As shown, in S202, during the voice conversation with the user, the real-time voice data of the user is obtained. The target user who meets the conditions can be screened out through the user screening strategy, and then a communication connection is established with the target user to conduct a voice conversation.

[0048] In the embodiments of the present disclosure, the speech strategy determination method of the present disclosure is described by taking the financial service scenario as an example, and subsequent user voice data, user data, etc. are also explained based on this. Without loss of generality, the speech strategy determination method for voice conversation in the present disclosure can be used in other fields and other usage scenarios, and the present disclosure will not go into details here.

[0049] A specific voice conversation scenario may be, for example, a post-loan collection scenario. Post-loan collection, as the last link in credit business risk management, is an important part of forming a closed loop of risk management. However, since post-loan transactions often involve direct dealings with customers, more uncertainties are involved. Therefore, how to scientifically manage post-loan collection is crucial. In the voice collection process, the administrator user and the user in arrears may have direct conversations for collection, and an intelligent robot may also be used to conduct intelligent voice conversations with the user for collection. In these two application scenarios, the speech strategy determination method in this application may be used to assist the intelligent robot or administrator in collection.

[0050] In S204, voice feature data of the real-time voice data is extracted based on the voice model to determine the emotion label of the user. The emotion label of the user may represent the emotion change of the user. The real-time voice data may be pre-processed to generate frame data; the frame data may be fast Fourier transformed to generate transformation data; the transformation data may be filtered through multiple Mel filter groups to generate voice feature data; the voice feature data may be input into the voice model to generate the emotion label of the user.

[0051] The speech model is trained by Mel-frequency cepstral coefficients and long short-term memory network. The specific training process is described in Figure 3 The details are described in the embodiments of the present invention.

[0052] Among them, the Mel-frequency cepstral coefficients (MFCC) are proposed based on the auditory characteristics of the human ear, and they have a nonlinear correspondence with the Hz frequency. The Mel-frequency cepstral coefficients (MFCC) use this relationship between them to calculate the Hz spectrum features. It is mainly used for voice data feature extraction and reducing the operation dimension. For example: for a frame with 512 dimensions (sampling points) of data, the most important 40 dimensions (generally speaking) of data can be extracted after MFCC, which also achieves the purpose of dimensionality reduction.

[0053] Long Short-Term Memory (LSTM) is a model of deep learning. Deep learning is the study of the inherent laws and representation levels of sample data. The information obtained in the learning process is very helpful for the interpretation of data such as text, images and sounds. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far exceed previous related technologies. A time-recurrent neural network, LSTM can be used as a complex nonlinear unit to construct a larger deep neural network.

[0054] In S206, semantic feature data of the real-time voice data is extracted based on the semantic model to determine the user's attitude label. The real-time voice data is converted into text data; the text data is processed to generate text feature data; the text feature data is input into the semantic model to generate the user's attitude label. The user's attitude label may represent the probability of the user repaying the loan.

[0055] The semantic model is trained by the recurrent neural network model. The specific training process is as follows: Figure 4 The details are described in the embodiments of the present invention.

[0056] Among them, the recurrent neural network model can be GRU, and GRU (Gate Recurrent Unit) is a kind of recurrent neural network (RNN). Like LSTM, it is also proposed to solve problems such as gradients in long-term memory and back propagation. Compared with LSTM, using GRU can achieve considerable results, and it is easier to train by comparison, which can greatly improve training efficiency. Compared with LSTM, GRU has less gating inside and fewer parameters than LSTM, but it can also achieve functions comparable to LSTM. Taking into account the computing power and time cost of hardware, the GRU model is selected in the embodiment of the present application.

[0057] In S208, a speech strategy is determined based on the emotion tag and the attitude tag. A corresponding speech strategy can be selected from pre-stored speech strategies according to the emotion tag and the attitude tag to conduct a subsequent voice call, and the speech strategy can be generated in advance based on experience data.

[0058] In one embodiment, the user emotion label may be: actively cooperate, and the attitude label may be: the probability of repayment is high. In this case, the speech strategy may be: soothing and coordination. In another embodiment, the user emotion label may be: bad and procrastination, and the attitude label may be: the probability of repayment is very low. In this case, the speech strategy may be: severe and urgent.

[0059] In one embodiment, for example, user data of the user may be obtained, the user data including basic data and behavior data; a user portrait of the user may be determined based on the user data; and the speech strategy may be determined based on the user portrait and the emotion label and the attitude label.

[0060] Among them, user data includes but is not limited to the user's business account information, the user's page operation data, the user's business access duration, the user's business access frequency, the user's terminal device identification information and the user's geographical location information, which can be determined according to the actual application scenario and is not limited here. User data may include basic data, such as business account information, user's terminal device identification information, user's geographical location information, etc.; user data may also include behavioral data, such as user's page operation data, user's business access duration, user's business access frequency, etc. The specific content of user data can be determined according to the actual application scenario and is not limited here. More specifically, the user information of the current user can be obtained based on user authorization by using web page embedding. Remote information may be user data of users on other trading platforms or other business departments.

[0061] More specifically, the user's behavior information on the website can be obtained through the Fiddler tool. The Fiddler tool works in the form of a web proxy server. After the client sends the request data, the Fiddler proxy server intercepts the data packet, and then the proxy server pretends to be the client to send data to the server; similarly, the server returns the response data, and the proxy server also intercepts the data and returns it to the client. Through Fiddler, you can obtain the user's web browsing residence time, residence page, click operation and other related browsing data.

[0062] The user portrait can be determined based on user data through machine learning models, and then the speech strategy can be determined for the user based on the characteristics of the user portrait.

[0063] More specifically, the dialogue strategy can be generated based on historical voice data. For example, voice data of successful debt collection in the past can be extracted, and the above voice data can be split into multiple segments according to the content of the conversation between the two parties. Emotional labels and attitude labels are assigned to each user's voice segment, and strategy labels are assigned to each intelligent robot voice segment (or administrator voice segment). Model training is performed on the above historical voice data to generate a dialogue strategy model. The dialogue strategy model can determine the most appropriate strategy for answering real-time voice data based on emotional labels and attitude labels.

[0064] In S210, reply data is determined based on the speech strategy to continue the voice conversation with the user, including: determining reply voice data or reply text data based on the speech tag; and having a voice conversation with the user based on the reply voice data or reply text data.

[0065] When the person having a voice call with a user is an administrator, the reply text data can be determined according to the speech strategy, which can be keywords or key reply content. The administrative user can have a voice call with the user based on the generated reply text.

[0066] When an intelligent robot is having a voice call with a user, the voice response data can be directly retrieved according to the speech strategy and the voice response data can be played to the user.

[0067] In the process of debt collection, the data that can fully represent both the user and the administrator (or intelligent robot) is the debt collection voice data. In the voice data, the user's tone, intonation and the emotional information behind it can establish a voice-based emotional model, which is used to further analyze the user's emotions and analyze the user's positive, negative, and neutral attitudes towards repayment. In addition, after the voice data is converted into text, the text information contained in it is also full of effective information. A text-based semantic model can be established based on this text to further evaluate the possibility of the user being recalled. The method for determining the speech strategy for voice dialogue in the present disclosure is mainly based on the debt collection voice data, and cross-analyzes from the two perspectives of voice and semantics, and then analyzes the user's emotions and the possibility of debt collection in real time, further improving the risk control ability of the post-loan system and reducing losses.

[0068] According to the method for determining speech strategy for voice conversation disclosed in the present invention, in the process of having a voice conversation with the user, the user's real-time voice data is obtained; based on the voice model, the voice feature data of the real-time voice data is extracted to determine the user's emotion label; based on the semantic model, the semantic feature data of the real-time voice data is extracted to determine the user's attitude label; the speech strategy is determined based on the emotion label and the attitude label; based on the speech strategy, the mode of replying data to continue the voice conversation with the user is determined, and the user's voice data is analyzed from two perspectives, namely, the voice and semantics. This can make full use of the information in the voice data and quickly and accurately determine the speech label, thereby improving the quality and accuracy of the voice conversation and improving user satisfaction.

[0069] It should be clearly understood that the present disclosure describes how to form and use specific examples, but the principles of the present disclosure are not limited to any details of these examples. On the contrary, based on the teachings of the contents disclosed in the present disclosure, these principles can be applied to many other embodiments.

[0070] Figure 3 The present invention is a flowchart of a method for determining a speech strategy for voice conversation according to another exemplary embodiment. Figure 3 The process 30 shown is a detailed description of “generating a speech model”.

[0071] like Figure 3 As shown, in S302, historical voice data of historical users is obtained. The log data of historical voice calls can be retrieved from the business database to obtain the historical voice data. The historical voice data can also be divided into multiple segments according to the actual time of the conversation.

[0072] In S304, an emotion tag is determined for the historical voice data. The emotion tag can be determined for the historical voice data manually. A tag can be set separately between each segment.

[0073] In S306, the long short-term memory network model is trained based on the historical speech data with emotion tags to generate the speech model. For example, historical speech feature data of the historical speech data can be extracted through Mel cepstral coefficients; the historical speech feature data with emotion tags is input into the long short-term memory network model for training; and the speech model is generated when the activation function softmax function satisfies the training strategy.

[0074] Specifically, the initial long short-term memory network model can be used as an adjustment model, and the historical speech data can be input into the adjustment model to obtain a predicted label. The predicted label is compared with the corresponding real label to determine whether the predicted label is consistent with the real label, and the number of predicted labels consistent with the real label is counted, and the proportion of the number of predicted labels consistent with the real label in the number of all predicted labels is calculated. If the proportion is greater than or equal to the preset proportion value, the adjustment model converges, and the adjustment model at this time is used as the trained speech model. If the proportion is less than the preset proportion value, the parameters in the adjustment model are adjusted, and the predicted labels of each historical speech data are re-predicted by the adjusted adjustment model until the proportion is greater than or equal to the preset proportion value. Among them, the method for adjusting the parameters in the adjustment model can be performed using a stochastic gradient descent algorithm, a gradient descent algorithm, or a normal equation. If the number of times the parameters of the adjustment model are adjusted exceeds the preset number, the model used to construct the adjustment model can be replaced to improve the model training efficiency.

[0075] Figure 4 The present invention is a flowchart of a method for determining a speech strategy for voice conversation according to another exemplary embodiment. Figure 4 The process 40 shown is a detailed description of “generating a semantic model”.

[0076] like Figure 4 As shown, in S402, historical voice data of historical users is obtained. The log data of historical voice calls can be retrieved from the business database to obtain the historical voice data. The historical voice data can also be divided into multiple segments according to the actual time of the conversation.

[0077] In S404, an attitude label is determined for the historical voice data. The attitude label can be determined for the historical voice data by whether the historical voice data actually repays the loan. If the user corresponding to the historical voice data actually repays the loan, the label is positive, otherwise it is negative. Labels can be set separately between each segment. The attitude label can also be determined for the historical voice data according to other preparations, and the present application is not limited thereto.

[0078] In S406, the recurrent neural network model is trained based on the historical speech data with attitude labels to generate the semantic model, including: converting the historical speech data into historical text data; generating historical text feature data through the historical text data; inputting the historical text feature data with attitude labels into the recurrent neural network model for training; and generating the semantic model when the activation function tanh function satisfies the training strategy.

[0079] The training process of the semantic model can refer to the training process of the speech data model, which will not be repeated here.

[0080] Figure 5 The present invention is a flowchart of a method for determining a speech strategy for voice conversation according to another exemplary embodiment. Figure 4 The process 40 shown is for Figure 2 Detailed description of the process shown.

[0081] like Figure 5 As shown, in S502, the call is connected.

[0082] In S504, real-time voice data is obtained. The system background transmits the voice to the model system in real time.

[0083] In S506, speech feature conversion is performed. The speech data is converted into a matrix that can be input into the model. In speech processing, commonly used speech features are digital features extracted using MFCC (Mel Cepstral Coefficients).

[0084] In S508, the speech model is input for calculation.

[0085] In one embodiment, the speech model LSTM establishment process is as follows:

[0086] 1. Collect historical voice data and manually mark the emotional attitude of the voice, which can be divided into three emotions: positive repayment, negative repayment, and neutral, as labels for the model.

[0087] 2. MFCC is used to extract speech features from historical speech data, combined with marked emotion tags, and the LSTM model is used to train the data, using softmax as the output activation function of the model.

[0088] 3. Deploy the trained model to the collection system. During collection, the converted output voice features in the voice data are obtained in real time as the input of the model. The model outputs the user's emotional preference. Since it is a softmax output, the emotional label with the highest probability is output.

[0089] In S510, voice is converted to text, using a common voice-to-text system and tool to convert between voice and text.

[0090] In S512, text feature conversion is performed on the text after speech conversion, such as word segmentation, onehot, embedding, etc., to process it into a numerical matrix for model input.

[0091] In S514, the semantic module is input for calculation.

[0092] In one embodiment, the speech model GRU establishment process is as follows:

[0093] 1. Collect historical voice data and convert it into text data. At the same time, mark it as 1 (recall) or 0 (no call) according to the result of the call.

[0094] 2. Perform feature extraction operations on the converted text data, such as word segmentation, one_hot, embedding, etc., and convert it into the model input matrix format. In order to be heterogeneous with the LSTM structure model in the speech model, a neural network based on the GRU structure is used to train the data to obtain a trained GRU model.

[0095] 3. Deploy the trained model to the collection system. During collection, the numerical matrix of the converted voice data is obtained in real time as the input of the model, and the model outputs the score of the user's ability to be collected.

[0096] In S516, real-time emotion tags and attitude tags are obtained. The collection method for the user is adjusted based on the emotion tags and attitude tags, such as changing the collection words, slowing down the collection tone, etc.

[0097] In S518, a speech strategy is determined to conduct the call.

[0098] The disclosed method for determining speech strategies for voice conversations uses MFCC to extract speech features based on real-time user speech data, and uses LSTM neural network to establish a speech-based sentiment analysis model. On the other hand, based on speech-to-text conversation text data, text features are extracted, and a text-based collection rate model is established using GRU neural network. The two types of scoring are combined to guide collection personnel to conduct effective collection.

[0099] The disclosed method for determining speech strategies for voice dialogue is based on voice data in the collection process, and models are established from two perspectives: voice and text, which are used for evaluating customer sentiment and scoring the likelihood of collection, respectively. On the one hand, by making full use of voice data, the efficiency of using voice data is improved, and the angles of using voice data are expanded. On the other hand, by characterizing and cross-using voice data from two angles, the post-loan risk management capabilities are effectively improved, the collection efficiency is improved, and the business risk is further reduced. In addition, model scoring and user sentiment analysis can be combined with other collection models as user portraits to further tap the potential of voice data.

[0100] Those skilled in the art will appreciate that all or part of the steps for implementing the above embodiments are implemented as a computer program executed by a CPU. When the computer program is executed by the CPU, the above functions defined by the above method provided by the present disclosure are performed. The program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0101] In addition, it should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0102] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0103] Figure 6 FIG. 1 is a block diagram of a device for determining a speech strategy for voice dialogue according to another exemplary embodiment. Figure 6 As shown, the speech strategy determination device 60 for voice dialogue includes: an acquisition module 602, a voice module 604, a semantic module 606, a strategy module 608, and a reply module 610.

[0104] The acquisition module 602 is used to acquire the user's real-time voice data during the voice conversation with the user;

[0105] The speech module 604 is used to extract speech feature data of the real-time speech data based on a speech model to determine the emotion label of the user;

[0106] The semantic module 606 is used to extract semantic feature data of the real-time speech data based on a semantic model to determine the attitude label of the user;

[0107] The strategy module 608 is used to determine the speech strategy based on the emotion tag and the attitude tag;

[0108] The reply module 610 is used to determine reply data based on the speech strategy to continue the voice conversation with the user.

[0109] According to the apparatus for determining speech strategy for voice conversation disclosed in the present invention, in the process of having a voice conversation with the user, real-time voice data of the user is acquired; based on a voice model, voice feature data of the real-time voice data is extracted to determine the emotion label of the user; based on a semantic model, semantic feature data of the real-time voice data is extracted to determine the attitude label of the user; a speech strategy is determined based on the emotion label and the attitude label; based on the speech strategy, a method of replying data to continue the voice conversation with the user is determined. The voice data of the user is analyzed from both the voice and semantic perspectives, and the information in the voice data can be fully utilized to quickly and accurately determine the speech label, thereby improving the quality and accuracy of the voice conversation and improving user satisfaction.

[0110] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment.

[0111] Refer to the following Figure 7 An electronic device 700 according to this embodiment of the present disclosure is described. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0112] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), a display unit 740, etc.

[0113] The storage unit stores a program code, which can be executed by the processing unit 710, so that the processing unit 710 performs the steps in this specification according to various exemplary embodiments of the present disclosure. For example, the processing unit 710 can perform the following steps: Figure 2 , Figure 3 , Figure 4 , Figure 5 Follow the steps shown in .

[0114] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 7201 and / or a cache memory unit 7202 , and may further include a read-only memory unit (ROM) 7203 .

[0115] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0116] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0117] The electronic device 700 may also communicate with one or more external devices 700' (e.g., keyboards, pointing devices, Bluetooth devices, etc.) to enable a user to communicate with the electronic device 700, and / or any device (e.g., routers, modems, etc.) that the electronic device 700 can communicate with one or more other computing devices. Such communication may be performed via an input / output (I / O) interface 750. In addition, the electronic device 700 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 760. The network adapter 760 may communicate with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0118] Through the above description of the implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by combining software with necessary hardware. Figure 8 As shown, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the embodiment of the present disclosure.

[0119] The software product may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0120] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0121] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0122] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by a device, the computer-readable medium realizes the following functions: in the process of conducting a voice conversation with the user, obtaining the user's real-time voice data; extracting the voice feature data of the real-time voice data based on the voice model to determine the user's emotion label; extracting the semantic feature data of the real-time voice data based on the semantic model to determine the user's attitude label; determining the speech strategy based on the emotion label and the attitude label; and determining the reply data based on the speech strategy to continue the voice conversation with the user.

[0123] Those skilled in the art will appreciate that the above modules can be distributed in the device according to the description of the embodiment, or can be changed accordingly and only used in one or more devices different from the embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

[0124] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0125] The exemplary embodiments of the present disclosure are specifically shown and described above. It should be understood that the present disclosure is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present disclosure is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A method for determining a speech strategy for voice dialogue, characterized in that: include: During a voice conversation with a user, obtaining real-time voice data of the user; Performing data preprocessing on the real-time voice data to generate frame data; Performing fast Fourier transform on the frame data to generate transformed data; Performing filtering processing on the transformed data through a plurality of Mel filter banks to generate speech feature data; Inputting the speech feature data into a speech model to generate an emotion label of the user; Convert real-time voice data into text data; Performing text processing on the text data to generate text feature data; Inputting the text feature data into a semantic model to generate an attitude label of the user; Obtaining user data of the user, wherein the user data includes basic data and behavior data; Determine a user profile of the user based on the user data; Determine a speech strategy based on the user portrait, the emotion tag, and the attitude tag; Determine response data based on the speech strategy to continue the voice conversation with the user.

2. The method according to claim 1, characterized in that Also includes: Get historical voice data of historical users; Determining an emotion label for the historical speech data; The long short-term memory network model is trained based on historical speech data with emotion labels to generate the speech model.

3. The method according to claim 2, characterized in that The long short-term memory network model is trained based on historical speech data with emotion labels to generate the speech model, including: Extracting historical speech feature data of the historical speech data through Mel-frequency cepstral coefficients; The historical speech feature data with emotion labels are input into the long short-term memory network model for training; When the activation function softmax function satisfies the training strategy, the speech model is generated.

4. The method according to claim 1, characterized in that Also includes: Get historical voice data of historical users; Determining an attitude label for the historical speech data; The recurrent neural network model is trained based on historical speech data with attitude labels to generate the semantic model.

5. The method according to claim 4, characterized in that The recurrent neural network model is trained based on the historical speech data with attitude labels to generate the semantic model, including: Converting the historical voice data into historical text data; Generate historical text feature data through the historical text data; Input historical text feature data with attitude labels into the recurrent neural network model for training; When the activation function tanh function satisfies the training strategy, the semantic model is generated.

6. The method according to claim 1, characterized in that Determining reply data based on the speech strategy to continue the voice conversation with the user includes: Determine reply voice data or reply text data based on the speech strategy; The voice dialogue with the user is continued based on the reply voice data or the reply text data.

7. A device for determining speech strategy for voice dialogue, characterized in that: include: The acquisition module is used to acquire the user's real-time voice data during the voice conversation with the user; A voice module, used for performing data preprocessing on the real-time voice data to generate frame data; performing fast Fourier transform on the frame data to generate transformed data; Performing filtering processing on the transformed data through a plurality of Mel filter banks to generate speech feature data; inputting the speech feature data into a speech model to generate an emotion label of the user; A semantic module, used for converting real-time voice data into text data; performing text processing on the text data to generate text feature data; and inputting the text feature data into a semantic model to generate an attitude label of the user; A policy module, used to obtain user data of the user, wherein the user data includes basic data and behavior data; Determine a user profile of the user based on the user data; Determine a speech strategy based on the user portrait, the emotion tag, and the attitude tag; A reply module is used to determine reply data based on the speech strategy to continue the voice conversation with the user.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.