Dialogue risk management and control method based on intelligent training risk coping strategy

Abstract

The invention discloses a dialogue risk management and control method based on an intelligent training risk coping strategy, and belongs to the technical field of artificial intelligence, and the method comprises the following steps: S1, when a mediator and a party start to communicate, a system automatically creates a scene canvas for each dialogue environment, identifying and capturing data generated in the call process in real time and adding the data to the scene canvas; s2, rating a risk level in real time by the system according to data in the scene canvas; s3, dynamically performing different interventions by the system according to different risk levels; and S4, the system synchronizes dynamic monitoring and intervention results to a quality inspection specialist, the quality inspection specialist performs manual scoring, the system learns the scoring result, optimizes and adjusts, and improves the judgment performance of the machine, in the invention, the system performs synchronous intervention on the video call, and the risk is controlled in the next second. The system intervenes in dialogues on the basis that risk assessment and coping strategies need to be trained, and the system needs to dynamically give the most appropriate risk prompting method or management and control means according to a training model.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a dialogue risk management method based on intelligent training risk response strategy. Background Art

[0002] At present, in most industries such as banking and communications, user emotion recognition is very important during telephone communication between customer service staff and users. Emotions can be used to predict user intentions and adjust the customer service staff's communication methods in a timely manner. In addition, clear emotions themselves can also be used as intentions for targeted responses.

[0003] The existing dialogue risk management technology has a lag. The system can only conduct risk assessment through traditional methods based on the saved data after the risk has occurred. Secondly, the backward risk control method relies on human training, and whether the effect is ideal depends to a large extent on the self-consciousness of the trainees, and the effect cannot be quantified and guaranteed.

[0004] Based on this, the present invention designs a dialogue risk management method based on intelligent training risk response strategy to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that the existing dialogue risk management technology has a lag, and the system can only perform risk assessment in a traditional way based on the saved data after the risk has occurred. Secondly, the risk control method is backward and relies on human training. Whether the effect is ideal depends to a large extent on the self-consciousness of the trainees, and the effect cannot be quantified and guaranteed. A dialogue risk management method based on intelligent training risk response strategy is proposed.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A dialogue risk management method based on intelligent training risk response strategy includes the following steps:

[0008] S1: When the mediator and the parties start talking, the system automatically creates a scene canvas for each conversation environment, and recognizes and captures the data generated during the conversation in real time and adds it to the scene canvas;

[0009] S2: The system rates the risk level in real time based on the data in the scenario canvas;

[0010] S3: The system dynamically intervenes differently for different risk levels;

[0011] S4: The system synchronizes the results of dynamic monitoring and intervention to the quality inspectors, who manually score them. The system learns the scoring results, optimizes and adjusts them, and improves the machine's judgment performance.

[0012] As a further description of the above technical solution:

[0013] The data includes emotions, pitch, volume, keywords and semantic vectors. The semantic vectors are composed of keywords into phrases or short sentences with language meanings, and the language meanings contain certain logical relationships.

[0014] As a further description of the above technical solution:

[0015] The data is determined by the system converting the call voice into text, identifying the keywords and phrases with voice, and querying the corresponding interpretation meaning to understand the semantics and logical relationship expressed. The emotion is determined by calling a large model to judge whether the emotion is irritable, calm or depressed according to the voice tone and facial information.

[0016] As a further description of the above technical solution:

[0017] The risk parameters are based on the data obtained in the integrated step S1 to determine the call risk level, and the risk parameters refer to keywords and emotions, and the keywords include words or phrases related to threats, insults, disdain and coercion.

[0018] As a further description of the above technical solution:

[0019] The relationship between the risk parameters is a positive correlation between the number frequency and the risk level, and the more risk keywords there are, the greater the risk.

[0020] As a further description of the above technical solution:

[0021] The risk level and the corresponding system intervention means are determined based on the risk level. The system dynamically monitors the dialogue scenario in real time and dynamically monitors the risk to minimize the effect.

[0022] As a further description of the above technical solution:

[0023] The large model is called to judge whether the emotion is irritable, calm or depressed according to the voice tone and face data:

[0024] Based on the IEMOCAP dataset, since there are a large number of high-frequency phrases that do not contain semantics in the IEMOCAP dataset, it is used to improve the calculation speed. At the same time, the frequency of phrases is counted, and phrases with less than 3 occurrences are deleted to reduce unnecessary vector modeling;

[0025] After determining the data set, use Glove to perform unsupervised training, extract call text features, define a skip-meta model, set the number of embedding layers to 10, the number of hidden layers to 2, and define the loss function sigmoid expression as:

[0026]

[0027] Import the network into the training module, set the learning rate to 0.002, and train for 60 rounds. After the model training is completed, output the three typical emotional word cloud graphs of irritability, calmness and depression in turn.

[0028] As a further description of the above technical solution:

[0029] In the process of call recording, there are environmental noise and equipment interference factors, so the original audio signal needs to be pre-emphasized, framed and blank frames removed in sequence. After a series of preprocessing, the original audio signal is obtained through fast Fourier transform and Mel cepstrum system to obtain time domain, spectral domain and cepstrum domain feature audio feature data;

[0030] A set of raw data corresponding to irritability, calmness, and depression is extracted from the IEMOCAP dataset, and the audio waveform is visualized. The pre-emphasis filter is completed by a first-order FIR high-pass digital filter, as shown in the following formula:

[0031] H(z)=1-az -1

[0032] Based on the multiplication technology of audio signals, the translation or conversion of the limited-length window function is realized, and the signal after windowing is:

[0033] sw(n)=s(n)*w(n)

[0034] In order to effectively retain the information content of the speech signal during its change process to the greatest extent possible, a sliding window is used. The speech sampling sequence formed after windowing is:

[0035] w(n)×s(n)→{s w |n=0,1,…,N-1}

[0036] After preprocessing, the image signal becomes smoother, eliminating noise and messy sound waves, making the features more prominent.

[0037] As a further description of the above technical solution:

[0038] Before collecting the facial data, a preliminary investigation is first conducted on how emotions affect people's behavior patterns, and the investigation results are used as the basis for collecting video call data;

[0039] Use an Android-based data collection tool to collect video call data in real time, and develop an emotion recording tool for system recording;

[0040] The collected multidimensional data are preprocessed, feature extracted, and feature fused.

[0041] As a further description of the above technical solution:

[0042] The feature embedding technology is used for modal fusion. In the feature embedding part, the output part of the face data and the speech feature extraction network is fused to form a multimodal feature vector matrix containing the emotion recognition result.

[0043] In order to make full use of the cross-modal information and time relationship between the modalities, the information of the two modalities is embedded. For each modality, the process of fusing multiple vectors from a single modality to form a multimodal vector is equivalent to the joint feature representation of the vector.

[0044] Six classifiers including support vector machine and two classification methods including basic emotion model and circumplex emotion model were used to perform emotion recognition and comparative verification on mixed data and personal data of multiple people.

[0045] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0046] 1. In the present invention, the system synchronously intervenes in the video call and controls the risk in the next second. The basis for the system to intervene in the conversation is to train risk assessment and response strategies. The system should dynamically give the most appropriate risk warning method or control means based on the training model.

[0047] 2. In the present invention, the results show that the synonymous phrases appearing in the word cloud diagrams corresponding to the three emotions are consistent with real-life experience, indicating that the trained Glove embedding model can effectively model the connections between emotional phrases.

[0048] 3. In the present invention, the Mel-graph corresponding to the audio emotion is extracted from the preprocessed audio waveform through the Mel-cephalogram coefficient, and the difference in short-time average energy of different audios can effectively distinguish different emotions, which is suitable for the emotional feature category of neural network learning, training and classification.

[0049] 4. In the present invention, in order to better obtain the emotional association between the front and back of the audio signal, the Transformer+CTC hybrid model is used to model the long-distance audio features, solve the problems of different input sequence lengths and difficulty in alignment, normalize the obtained waveform and Mel map, subsample the audio to reduce the sequence length, and then process the Mel map through a convolutional network, use the CTC algorithm to achieve sequence alignment, and finally convert it into an embedded sample with a dimension of 512. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a dialogue risk management method based on an intelligent training risk response strategy proposed by the present invention;

[0051] Figure 2 A schematic diagram of the audio feature extraction process in a conversation risk management method based on an intelligent training risk response strategy proposed by the present invention;

[0052] Figure 3 This is a schematic diagram of the framework structure of emotion recognition in a dialogue risk management method based on an intelligent training risk response strategy proposed in the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Please see attached Figure 1-3 The present invention provides a technical solution: a dialogue risk management method based on intelligent training risk response strategy, comprising the following steps:

[0055] S1: When the mediator and the parties start talking, the system automatically creates a scene canvas for each dialogue environment, and recognizes and captures the data generated during the call in real time and adds it to the scene canvas. The data includes emotions, pitch, volume, keywords and semantic vectors. The semantic vectors are composed of keywords into phrases or short sentences with language meanings, and the language meanings contain certain logical relationships. The data is determined by the system converting the call voice into text, identifying the keywords and phrases with voice, and querying the corresponding interpretation meanings to understand the semantics and logical relationships expressed. The determination of emotions is to call a large model to judge whether the emotions are irritable, calm or depressed according to the voice tone and facial information;

[0056] The large model is called to judge whether the emotion is irritable, calm or depressed according to the voice tone and face data:

[0057] Based on the IEMOCAP dataset, since there are a large number of high-frequency phrases that do not contain semantics in the IEMOCAP dataset, it is used to improve the calculation speed. At the same time, the frequency of phrases is counted, and phrases with less than 3 occurrences are deleted to reduce unnecessary vector modeling;

[0058] After determining the data set, use Glove to perform unsupervised training, extract call text features, define a skip-meta model, set the number of embedding layers to 10, the number of hidden layers to 2, and define the loss function sigmoid expression as:

[0059]

[0060] Import the network into the training module and set the learning rate to 0.002. Train for 60 rounds. After the model training is completed, output the three typical emotional word clouds of irritability, calmness and depression in turn.

[0061] In the process of call recording, there are environmental noise and equipment interference factors, so the original audio signal needs to be pre-emphasized, framed and blank frames removed in sequence. After a series of preprocessing, the original audio signal is obtained through fast Fourier transform and Mel cepstrum system to obtain time domain, spectral domain and cepstrum domain feature audio feature data;

[0062] A set of raw data corresponding to irritability, calmness, and depression is extracted from the IEMOCAP dataset, and the audio waveform is visualized. The pre-emphasis filter is completed by a first-order FIR high-pass digital filter, as shown in the following formula:

[0063] H(z)=1-az -1

[0064] Based on the multiplication technology of audio signals, the translation or conversion of the limited-length window function is realized, and the signal after windowing is:

[0065] s w (n) = s(n) * w(n)

[0066] In order to effectively retain the information content of the speech signal during its change process to the greatest extent possible, a sliding window is used. The speech sampling sequence formed after windowing is:

[0067] w(n)×s(n)→{s w |n=0,1,…,N-1}

[0068] After preprocessing, the image signal becomes smoother, eliminating noise and messy sound waves, making the features more prominent;

[0069] Before collecting the facial data, a preliminary investigation is first conducted on how emotions affect people's behavior patterns, and the investigation results are used as the basis for collecting video call data;

[0070] Use an Android-based data collection tool to collect video call data in real time, and develop an emotion recording tool for system recording;

[0071] Preprocessing, feature extraction and feature fusion of the collected multidimensional data;

[0072] S2: The risk parameters are based on the data obtained in the comprehensive step S1 to determine the call risk level. The risk parameters refer to keywords and emotions. The keywords include words or phrases related to threats, insults, disdain and coercion. The relationship between the risk parameters is a positive correlation between the number frequency and the risk level. The more risk keywords, the greater the risk. The risk level and the corresponding system intervention means are determined based on the risk level. The system dynamically monitors the conversation scenario in real time and dynamically monitors the effect of minimizing the risk. The system rates the risk level in real time according to the data in the scenario canvas. The risk level and the corresponding system intervention means are as follows:

[0073]

[0074]

[0075] S3: The system dynamically intervenes differently for different risk levels;

[0076] S4: The system synchronizes the results of dynamic monitoring and intervention to the quality inspection specialists, who manually score them. The system learns the scoring results, makes optimization adjustments, and improves the machine judgment performance.

[0077]

[0078]

[0079]

[0080] Specifically, the data includes emotions, pitch, volume, keywords and semantic vectors, wherein the semantic vectors are composed of keywords into phrases or short sentences with language meanings, and the language meanings contain certain logical relationships;

[0081] The feature embedding technology is used for modal fusion. In the feature embedding part, the output part of the face data and the speech feature extraction network is fused to form a multimodal feature vector matrix containing the emotion recognition result.

[0082] In order to make full use of the cross-modal information and time relationship between the modalities, the information of the two modalities is embedded. For each modality, the process of fusing multiple vectors from a single modality to form a multimodal vector is equivalent to the joint feature representation of the vector.

[0083] Six classifiers including support vector machine and two classification methods including basic emotion model and circumplex emotion model were used to perform emotion recognition and comparative verification on mixed data and personal data of multiple people.

[0084] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A dialogue risk management method based on intelligent training risk response strategy, characterized in that: The following steps are involved: S1: When the mediator and the parties start talking, the system automatically creates a scene canvas for each conversation environment, and recognizes and captures the data generated during the conversation in real time and adds it to the scene canvas; S2: The system rates the risk level in real time based on the data in the scenario canvas; S3: The system dynamically intervenes differently for different risk levels; S4: The system synchronizes the results of dynamic monitoring and intervention to the quality inspectors, who manually score them. The system learns the scoring results, optimizes and adjusts them, and improves the machine's judgment performance.

2. According to claim 1, a dialogue risk management method based on intelligent training risk response strategy is characterized in that: The data includes emotions, pitch, volume, keywords and semantic vectors. The semantic vectors are composed of keywords into phrases or short sentences with language meanings, and the language meanings contain certain logical relationships.

3. According to claim 2, a dialogue risk management method based on intelligent training risk response strategy is characterized in that: The data is determined by the system converting the call voice into text, identifying the keywords and phrases with voice, and querying the corresponding interpretation meaning to understand the semantics and logical relationship expressed. The emotion is determined by calling a large model to judge whether the emotion is irritable, calm or depressed according to the voice tone and facial information.

4. According to claim 1, a method for managing and controlling dialogue risks based on intelligent training risk response strategies is characterized in that: The risk parameters are based on the data obtained in the integrated step S1 to determine the call risk level, and the risk parameters refer to keywords and emotions, and the keywords include words or phrases related to threats, insults, disdain and coercion.

5. According to claim 4, a method for managing and controlling dialogue risks based on intelligent training risk response strategies is characterized in that: The relationship between the risk parameters is a positive correlation between the number frequency and the risk level, and the more risk keywords there are, the greater the risk.

6. According to claim 1, a method for managing and controlling dialogue risks based on intelligent training risk response strategies is characterized in that: The risk level and the corresponding system intervention means are determined based on the risk level. The system dynamically monitors the dialogue scenario in real time and dynamically monitors the risk to minimize the effect.

7. According to claim 1, a method for managing and controlling dialogue risks based on intelligent training risk response strategies is characterized in that: The large model is called to judge whether the emotion is irritable, calm or depressed according to the voice tone and face data: Based on the IEMOCAP dataset, since there are a large number of high-frequency phrases that do not contain semantics in the IEMOCAP dataset, it is used to improve the calculation speed. At the same time, the frequency of phrases is counted, and phrases with less than 3 occurrences are deleted to reduce unnecessary vector modeling; After determining the data set, use Glove to perform unsupervised training, extract call text features, define a skip-meta model, set the number of embedding layers to 10, the number of hidden layers to 2, and define the loss function sigmoid expression as: Import the network into the training module, set the learning rate to 0.002, and train for 60 rounds. After the model training is completed, output the three typical emotional word cloud graphs of irritability, calmness and depression in turn.

8. The method for dialogue risk management based on intelligent training risk response strategy according to claim 7 is characterized in that: In the process of call recording, there are environmental noise and equipment interference factors, so the original audio signal needs to be pre-emphasized, framed and blank frames removed in sequence. After a series of preprocessing, the original audio signal is obtained through fast Fourier transform and Mel cepstrum system to obtain time domain, spectral domain and cepstrum domain feature audio feature data; A set of raw data corresponding to irritability, calmness, and depression is extracted from the IEMOCAP dataset, and the audio waveform is visualized. The pre-emphasis filter is completed by a first-order FIR high-pass digital filter, as shown in the following formula: H(z)=1-az -1 Based on the multiplication technology of audio signals, the translation or conversion of the limited-length window function is realized, and the signal after windowing is: s w (n)=s(n)*w(n) In order to effectively retain the information content of the speech signal during its change process to the greatest extent possible, a sliding window is used. The speech sampling sequence formed after windowing is: w(n)×s(n)→{s w |n=0,1,…,N-1} After preprocessing, the image signal becomes smoother, eliminating noise and messy sound waves, making the features more prominent.

9. A method for managing and controlling dialogue risks based on intelligent training risk response strategies according to claim 8, characterized in that: Before collecting the facial data, a preliminary investigation is first conducted on how emotions affect people's behavior patterns, and the investigation results are used as the basis for collecting video call data; Use an Android-based data collection tool to collect video call data in real time, and develop an emotion recording tool for system recording; The collected multidimensional data are preprocessed, feature extracted, and feature fused.

10. A method for managing and controlling dialogue risks based on intelligent training risk response strategies according to claim 9, characterized in that: The feature embedding technology is used for modal fusion. In the feature embedding part, the output part of the face data and the speech feature extraction network is fused to form a multimodal feature vector matrix containing the emotion recognition result. In order to make full use of the cross-modal information and time relationship between the modalities, the information of the two modalities is embedded. For each modality, the process of fusing multiple vectors from a single modality to form a multimodal vector is equivalent to the joint feature representation of the vector. Six classifiers including support vector machine and two classification methods including basic emotion model and circumplex emotion model were used to perform emotion recognition and comparative verification on mixed data and personal data of multiple people.

Images