Model training methods, methods for predicting escalating complaint behavior, and related equipment

Abstract

This application provides a model training method, an escalation complaint behavior prediction method, and related equipment, applied in the field of artificial intelligence technology. The model training method includes: acquiring multiple training data sets and training an initial model, which includes an initial emotion recognition model, an initial sentiment analysis model, and an initial complaint classification model. The initial emotion recognition model is used to: receive historical complaint voice data, perform emotion recognition on the historical complaint voice data, and output the emotion recognition result. The initial sentiment analysis model is used to: receive historical complaint voice data, perform sentiment tendency analysis on the historical complaint voice data, and output the sentiment tendency result. The initial complaint classification model is used to: receive historical user profile data, predict whether a user will escalate their complaint based on the historical user profile data, and output the complaint escalation prediction result. The target model trained by this method can improve the accuracy of predicting whether a user will escalate their complaint behavior.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training method, an escalation complaint behavior prediction method, and related equipment. Background Technology

[0002] With the development of mobile internet, the requirements for user satisfaction perception are getting higher and higher. Among them, predicting whether users will escalate their complaints after receiving them, and taking corresponding countermeasures based on the prediction results, plays an important role in improving the user service experience.

[0003] In existing technologies, users' historical behavior data, such as the number of inquiries and complaints in the past 7 days, are generally used to predict whether users will escalate their complaints. This static analysis approach ignores the correlation between users' real-time behavior and whether they will escalate their complaints, resulting in low accuracy of the prediction results. Summary of the Invention

[0004] This application provides a model training method, an escalation complaint behavior prediction method, and related equipment, which can solve the problem of low accuracy in predicting whether a user will escalate their complaint behavior in the prior art.

[0005] In a first aspect, embodiments of this application provide a model training method, the method comprising:

[0006] Multiple training data sets are acquired, including historical user profile data, historical complaint voice data corresponding to the historical user profile data, emotion tags corresponding to the historical complaint voice data, sentiment tendency tags corresponding to the historical complaint voice data, and complaint escalation tags corresponding to the historical user profile data. The complaint escalation tags are used to indicate whether the user corresponding to the historical user profile data should escalate the complaint when the user sends the voice corresponding to the historical complaint voice data.

[0007] Based on the multiple training data sets, an initial model is trained to obtain a target model. The initial model includes an initial emotion recognition model, an initial sentiment analysis model, an initial complaint classification model, and an initial prediction model. The initial emotion recognition model is used to: receive the historical complaint voice data, perform emotion recognition on the historical complaint voice data, and output an emotion recognition result. The initial sentiment analysis model is used to: receive the historical complaint voice data, perform sentiment tendency analysis on the historical complaint voice data, and output a sentiment tendency result. The initial complaint classification model is used to: receive the historical user profile data, predict whether a user will escalate their complaint based on the historical user profile data, and output a complaint escalation prediction result. The initial prediction model is used to: receive the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determine a complaint escalation score based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation score indicates the probability of the corresponding user escalating their complaint.

[0008] Optionally, training the initial model based on the plurality of training data sets to obtain the target model includes:

[0009] The historical complaint voice data in the plurality of training data groups are preprocessed to obtain a first voice data set. The first voice data set includes all historical complaint voice data after preprocessing in the plurality of training data groups. The voice data preprocessing includes at least one of the following: noise reduction, frame segmentation, windowing, and length alignment.

[0010] Each historical complaint voice data after preprocessing in the first voice data set is converted into text to obtain a first text feature set, which includes all text features after text conversion in the first voice data set.

[0011] The historical user profile data in the plurality of training data groups are preprocessed to obtain a first profile data set. The first profile data set includes all historical user profile data after the profile data preprocessing in the plurality of training data groups. The profile data preprocessing includes at least one of the following: handling missing values ​​of numerical features, handling missing values ​​of categorical features, handling extreme missing values, and normalization.

[0012] Based on the emotion tags in the first speech data set, the first text feature set, and the plurality of training data sets, the initial emotion recognition model is trained to obtain a target emotion recognition model; and based on the sentiment tendency tags in the first text feature set and the plurality of training data sets, the initial sentiment analysis model is trained to obtain a target sentiment analysis model; and based on the complaint escalation tags in the first profile data set and the plurality of training data sets, the initial complaint classification model is trained to obtain a target complaint classification model; wherein, the target model includes the target emotion recognition model, the target sentiment analysis model, and the target complaint classification model.

[0013] Optionally, training the initial emotion recognition model based on the emotion labels in the first speech data set, the first text feature set, and the plurality of training data sets to obtain the target emotion recognition model includes: performing text feature preprocessing on the text features in the first text feature set to obtain a second text feature data set, wherein the text feature preprocessing includes at least one of the following: text cleaning, semantic encoding; performing a first enhancement process on at least one preprocessed historical complaint speech data in the first speech data set to obtain at least one enhanced speech data corresponding one-to-one with the at least one preprocessed historical complaint speech data, wherein the first enhancement process includes at least one of the following: adding background noise. The process involves: fundamental frequency transformation; adding the at least one enhanced speech data to the first speech data set to obtain a second speech data set; performing a second enhancement process on at least one first text feature in the second text feature set to obtain at least one first enhanced text feature corresponding one-to-one with the at least one first text feature, wherein the second enhancement process includes at least one of the following: synonym replacement, random insertion or deletion of lexical units; adding the at least one first enhanced text feature to the second text feature set to obtain a third text feature set; and training the initial emotion recognition model based on the second speech data set, the third text feature set, and the emotion labels in the plurality of training data sets to obtain the target emotion recognition model.

[0014] And / or, training the initial sentiment analysis model based on the first text feature set and the sentiment tendency labels in the plurality of training data groups to obtain the target sentiment analysis model includes: performing a first preprocessing on the text features in the first text feature set to obtain a fourth text feature set, wherein the first preprocessing includes at least one of the following: text clarity, dialogue structuring, and format conversion; performing a third enhancement processing on at least one of the first preprocessed first text features in the fourth text feature set to obtain at least one second enhanced text feature corresponding one-to-one with the at least one first preprocessed first text feature, wherein the third enhancement processing includes at least one of the following: synonym sentiment word replacement, back-translation enhancement, and generative enhancement; adding the at least one second enhanced text feature to the fourth text feature set to obtain a fifth text feature set; and training the initial sentiment analysis model based on the fifth text feature set and the sentiment tendency labels in the plurality of training data groups to obtain the target sentiment analysis model.

[0015] And / or, training the initial complaint classification model based on the complaint escalation labels in the first profile data set and the plurality of training data groups to obtain the target complaint classification model includes: generating virtual samples from at least one first profile data in the first profile data set to obtain at least one first enhanced profile data, wherein the complaint escalation label corresponding to the first profile data is a first label, which is used to indicate that the user has escalated the complaint; adding the at least one first enhanced profile data to the first profile data set to obtain a second profile data set; deleting at least one second profile data from the second profile data set to obtain a third profile data set, wherein the complaint escalation label corresponding to the second profile data is a second label, and the first label is used to indicate that the user has not escalated the complaint; and training the initial complaint classification model based on the third profile data set and the complaint escalation labels in the plurality of training data groups to obtain the target complaint classification model.

[0016] Optionally, the initial emotion recognition model includes a cross-attention interaction layer, a multimodal fusion layer, and an emotion classification layer;

[0017] The cross-attention interaction layer is used to: enhance text features using speech features to obtain text-enhanced features, and enhance speech features using the text features to obtain speech-enhanced features, wherein the text features are features obtained by text conversion of complaint speech data input into the initial emotion recognition model, and the speech features are features obtained by acoustic feature extraction of complaint speech data input into the initial emotion recognition model.

[0018] The multimodal fusion layer is used to: non-linearly integrate the text enhancement features and the speech enhancement features to obtain fused features;

[0019] The emotion classification layer is used to: perform emotion recognition on the fused features and output the emotion recognition result.

[0020] Secondly, embodiments of this application also provide a method for predicting escalation complaint behavior, the method comprising:

[0021] The target user's complaint voice data and user profile data are input into a target model to obtain the target model's output of emotion recognition results, sentiment tendency results, complaint escalation prediction results, and complaint escalation score. The target model is trained using the model training method described in the first aspect. The target model includes a target emotion recognition model, a target sentiment analysis model, a target complaint classification model, and a target prediction model. The target emotion recognition model receives the complaint voice data, performs emotion recognition on the complaint voice data, and outputs the emotion recognition result. The target sentiment analysis model receives the complaint voice data, performs sentiment tendency analysis on the complaint voice data, and outputs the sentiment tendency result. The target complaint classification model receives the user profile data, predicts whether the user will escalate their complaint based on the user profile data, and outputs the complaint escalation prediction result. The target prediction model receives the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determines the target user's complaint escalation score based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation prediction score indicates the probability that the target user will escalate their complaint.

[0022] Optionally, the emotion recognition result includes an emotion recognition score, the sentiment tendency result includes a sentiment tendency score, and the complaint escalation prediction result includes a complaint escalation prediction score.

[0023] The step of determining the complaint escalation score of the target user based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result includes:

[0024] The emotion recognition score, the sentiment tendency score, and the complaint escalation prediction score are normalized to obtain a first normalized value corresponding to the emotion recognition score, a second normalized value corresponding to the sentiment tendency score, and a third normalized value corresponding to the complaint escalation prediction score.

[0025] Using preset weight information, the first normalized value, the second normalized value, and the third normalized value are weighted and summed to obtain the complaint escalation score. The preset weight information is used to indicate: a first weight corresponding to the emotion recognition score, a second weight corresponding to the emotion tendency score, and a third weight corresponding to the complaint escalation prediction score. The sum of the first weight, the second weight, and the third weight is 1.

[0026] Secondly, embodiments of this application also provide an escalation complaint behavior prediction device, the device comprising:

[0027] The acquisition module is used to acquire multiple training data sets, which include historical user profile data, historical complaint voice data corresponding to the historical user profile data, emotion tags corresponding to the historical complaint voice data, sentiment tendency tags corresponding to the historical complaint voice data, and complaint escalation tags corresponding to the historical user profile data. The complaint escalation tags are used to indicate whether the user corresponding to the historical user profile data should escalate the complaint when the user sends the voice corresponding to the historical complaint voice data.

[0028] The training module is used to train an initial model based on the multiple training data sets to obtain a target model. The initial model includes an initial emotion recognition model, an initial sentiment analysis model, an initial complaint classification model, and an initial prediction model. Specifically, the initial emotion recognition model receives historical complaint voice data, performs emotion recognition on the historical complaint voice data, and outputs an emotion recognition result. The initial sentiment analysis model receives historical complaint voice data, performs sentiment tendency analysis on the historical complaint voice data, and outputs a sentiment tendency result. The initial complaint classification model receives historical user profile data, predicts whether a user will escalate their complaint based on the historical user profile data, and outputs a complaint escalation prediction result. The initial prediction model receives the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determines a complaint escalation score based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation score indicates the probability of the corresponding user escalating their complaint.

[0029] The device further includes:

[0030] The prediction module is used to input the complaint voice data of the target user and the user profile data of the target user into the target model, and obtain the emotion recognition result, sentiment tendency result, complaint escalation prediction result, and complaint escalation score output by the target model. The target model is a model trained based on the model training method described in the first aspect. The target model includes a target emotion recognition model, a target sentiment analysis model, a target complaint classification model, and a target prediction model. The target emotion recognition model is used to: receive the complaint voice data, perform emotion recognition on the complaint voice data, and output the emotion recognition result. The target sentiment analysis model is used to: receive the complaint voice data, perform sentiment tendency analysis on the complaint voice data, and output the sentiment tendency result. The target complaint classification model is used to: receive the user profile data, predict whether the user will escalate the complaint based on the user profile data, and output the complaint escalation prediction result. The target prediction model is used to: receive the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determine the complaint escalation score of the target user based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation prediction score is used to indicate the probability that the target user will escalate the complaint.

[0031] Thirdly, embodiments of this application also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the model training method as described in the first aspect, or when executed by the processor, it implements the steps of the escalation complaint behavior prediction method as described in the second aspect.

[0032] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the model training method as described in the first aspect, or, when executed by the processor, implements the steps of the escalation complaint behavior prediction method as described in the second aspect.

[0033] Fifthly, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the model training method as described in the first aspect, or, when executed by the processor, implement the steps of the escalation complaint behavior prediction method as described in the second aspect.

[0034] In this embodiment, the training data set includes historical user profile data, historical complaint voice data corresponding to the historical user profile data, emotion tags corresponding to the historical complaint voice data, sentiment tendency tags corresponding to the historical complaint voice data, and complaint escalation tags corresponding to the historical user profile data. The initial model includes an initial emotion recognition model, an initial sentiment analysis model, an initial complaint classification model, and an initial prediction model. Multiple training data sets are used to train the initial model to obtain the target model. This target model can output emotion recognition results and sentiment tendency results based on the user's complaint voice data, and can also output complaint escalation prediction results based on the user's user profile data. Since the emotion recognition results and sentiment tendency results are derived from analyzing the user's real-time behavior, and the escalation prediction results are derived from analyzing the user's historical behavior, the target model can simultaneously consider the impact of the user's real-time and historical behavior on whether the user will escalate the complaint. This improves the accuracy of the obtained complaint escalation score, which is equivalent to improving the prediction accuracy of whether the user will escalate the complaint. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is the flow chart of the model training method provided in the embodiments of this application. Figure 1 ;

[0037] Figure 2 This is the flow chart of the model training method provided in the embodiments of this application. Figure 2 ;

[0038] Figure 3 This is a schematic diagram of the speech feature extraction model network provided in the embodiments of this application;

[0039] Figure 4 This is a schematic diagram of the text feature extraction model network provided in the embodiments of this application;

[0040] Figure 5 This is a schematic diagram of the structure of the initial emotion recognition model provided in the embodiments of this application;

[0041] Figure 6 This is a flowchart of the escalation complaint behavior prediction method provided in the embodiments of this application;

[0042] Figure 7 This is a structural diagram of the escalation complaint behavior prediction device provided in the embodiments of this application;

[0043] Figure 8 This is a structural diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] This application provides a model training method, an escalation complaint behavior prediction method, and related equipment.

[0046] See Figure 1 , Figure 1 This is a flowchart of the model training method provided in the embodiments of this application, such as... Figure 1 As shown, it includes the following steps:

[0047] Step 101: Obtain multiple training data sets, which include historical user profile data, historical complaint voice data corresponding to the historical user profile data, emotion tags corresponding to the historical complaint voice data, sentiment tendency tags corresponding to the historical complaint voice data, and complaint escalation tags corresponding to the historical user profile data. The complaint escalation tags are used to indicate whether the user corresponding to the historical user profile data should escalate the complaint when the user sends the voice corresponding to the historical complaint voice data.

[0048] Step 102: Based on the multiple training data sets, train the initial model to obtain the target model. The initial model includes an initial emotion recognition model, an initial sentiment analysis model, an initial complaint classification model, and an initial prediction model. The initial emotion recognition model is used to: receive the historical complaint voice data, perform emotion recognition on the historical complaint voice data, and output an emotion recognition result. The initial sentiment analysis model is used to: receive the historical complaint voice data, perform sentiment tendency analysis on the historical complaint voice data, and output a sentiment tendency result. The initial complaint classification model is used to: receive the historical user profile data, predict whether a user will escalate their complaint based on the historical user profile data, and output a complaint escalation prediction result. The initial prediction model is used to: receive the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determine a complaint escalation score based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation score indicates the probability of the corresponding user escalating their complaint.

[0049] The target model trained by the model training method in this application is used to predict the probability of a user's escalation complaint behavior, where escalation complaint behavior refers to:

[0050] This refers to the act of a user escalating their complaint to a higher level or a more influential channel after initially filing a complaint through ordinary channels. For example, escalating from a complaint to customer service to a supervisor / manager, or from an internal company complaint to a regulatory body.

[0051] In this embodiment, the complaint escalation tag is used to indicate whether the user will file a complaint within the next n days after sending the complaint voice message corresponding to the historical complaint voice data (1 = complaint, 0 = no complaint). Historical user profile data can include 102 indicators in three main categories: basic attributes, business behavior, and complaint-related data, as shown in the table below:

[0052] Indicator Categories Indicator Name Basic attributes Age, gender, network tenure, user rating, main planner's name, whether there are repeated complaints, whether the user is a broadband customer, whether the user is a corporate user, whether the user is on a campus network, whether the user is on a home network, whether the user has a lucky number, whether the user is a ported-in user, whether the user is on a 4G plan, whether the user is a sensitive user in a certain city, terminal price, whether the user is a sensitive user of an online company. Business activities ARPU (RMB) last month, ARPU (RMB) two months ago, ARPU growth rate (2 months), ARPU growth rate (3 months), whether there was a tariff upgrade, whether there was a tariff downgrade, excess data charges (RMB), DOU (MB) last month, DOU (MB) two months ago, MOU (minutes) last month, MOU (minutes) two months ago, single suspension days this month, double suspension days this month, international roaming call percentage this month, local call percentage this month, weekend call percentage this month, international roaming data usage percentage this month, intra-provincial data usage percentage this month, weekend data usage percentage this month, 4G data usage percentage this month, peak hour data usage percentage this month, off-peak hour data usage percentage this month, total SMS messages this month, voice call charges this month, data usage charges this month, outstanding amount this month, number of payments this month, payment amount this month, voice saturation this month, data usage saturation this month, percentage of excessive voice calls this month, percentage of excessive data usage this month, percentage of excessive charges this month, whether it is a 4G terminal, base value of the main plan (RMB). Complaint related Narrowly defined complaint recordings, narrowly defined complaint recordings transcribed into text (full text), narrowly defined complaint type, whether it's a duplicate complaint (broadly defined duplicate complaints within the last three months, regardless of complaint type), whether it's a duplicate complaint (narrowly defined duplicate complaints within the last three months, and the complaint type is consistent), processing time (hours), resolution satisfaction, number of ordinary complaints within 3 months (excluding the current month), number of most common ordinary complaints within 3 months (excluding the current month), number of ordinary complaints within 6 months (excluding the current month), number of most common ordinary complaints within 6 months (excluding the current month), number of broadly defined complaints within 3 months (excluding the current month), and so on. The most frequent type of complaints (excluding the current month), the number of broad complaints within 6 months (excluding the current month), the most frequent type of broad complaints within 6 months (excluding the current month), the number of escalated complaints within 3 months (excluding the current month), the most frequent type of escalated complaints within 3 months (excluding the current month), the number of escalated complaints within 6 months (excluding the current month), the number of escalated complaints within 6 months (excluding the current month), whether there was compensation after the complaint, the last type of narrow-sense complaint, the average processing time of the previous three narrow-sense complaints, the average call center involvement processing time of the previous three narrow-sense complaints, and the average city-level involvement processing time of the previous three narrow-sense complaints. The following metrics are considered: length of time, average provincial department handling time for the previous three narrow-sense complaints, average number of complaint work order transfers for the previous three narrow-sense complaints, processing time of the previous narrow-sense complaint work order, call center handling time for the previous narrow-sense complaint, city-level handling time for the previous narrow-sense complaint, provincial department handling time for the previous narrow-sense complaint, number of complaint work order transfers for the previous narrow-sense complaint, satisfaction rate of follow-up visits for the previous narrow-sense complaint, resolution rate of follow-up visits for the previous narrow-sense complaint, whether the previous narrow-sense complaint was escalated, time elapsed between the previous narrow-sense complaint and the current narrow-sense complaint, historical cumulative number of narrow-sense complaints, type of complaint escalated last time, and previous escalation. Complaint processing time, call center involvement time for the last escalated complaint, city-level involvement time for the last escalated complaint, provincial-level involvement time for the last escalated complaint, number of times the complaint was transferred for the last escalated complaint, satisfaction rate of follow-up calls for the last escalated complaint, resolution rate of follow-up calls for the last escalated complaint, time elapsed between the last escalated complaint and the current complaint, whether the current escalated complaint is a duplicate, historical cumulative number of escalated complaints, number of calls to 12315 in the last three months, number of calls to 10086 in the last three months, number of calls to 10000 in the last three months, and number of calls to 10010 in the last three months.

[0053] In this embodiment, the historical complaint voice data corresponding to the historical user profile data refers to the voice data corresponding to the most recent complaint voice issued by the target user corresponding to the historical user profile data before the time of acquiring the multiple training data sets. For example, if the time of acquiring the multiple training data sets is the first day of month t, then the historical complaint voice data is the voice data corresponding to the latest complaint voice issued by the target user before the first day of month t. It can be understood that the target user may have issued many complaint voices before the first day of month t, but in this embodiment, the historical complaint voice data specifically refers to the data corresponding to the most recently issued complaint voice. Because the escalation complaint behavior prediction method in this embodiment associates the "real-time emotion of the user when making a complaint" with "whether the user will escalate the complaint," and the user's latest complaint voice data best reflects the user's real-time emotion when making a complaint, the historical complaint voice data used in the initial model training is the aforementioned voice data.

[0054] In this embodiment, the emotion tag is used to indicate the specific emotion of the user corresponding to the historical user profile data when issuing the voice message corresponding to the historical complaint voice message data, such as happiness, anger, sadness, etc.; the sentiment tendency tag is used to indicate the sentiment tendency of the user corresponding to the historical user profile data when issuing the voice message corresponding to the historical complaint voice message data, such as neutral sentiment tendency, negative sentiment tendency, positive sentiment tendency, etc. It should be noted that the emotion tag is used to indicate a specific emotion, while the sentiment tendency tag is used to indicate a more generalized sentiment tendency. Combining the two can more accurately determine the user's real-time emotion when issuing the complaint voice message data. Understandably, if the user's real-time emotion is negative or angry, then the user is more likely to escalate the complaint.

[0055] The historical complaint voice data, emotion tags, and sentiment tendency tags can be obtained by: exporting call recordings from business systems such as customer service systems and call centers, ensuring that each recording covers the entire call process and has no serious noise in the sound quality; and labeling each recording with the following content: emotion tendency (e.g., S represents positive emotion, D represents negative emotion), emotion category (e.g., anger, sadness), and text semantics (intention of the appeal, such as complaint, refund, etc.).

[0056] The initial model includes an initial emotion recognition model, an initial sentiment analysis model, an initial complaint classification model, and an initial prediction model. The initial emotion recognition and initial sentiment analysis models are used to analyze historical complaint voice data (representing real-time user behavior when making a complaint), obtaining emotion recognition and sentiment tendency results for the user corresponding to the historical user profile data when uttering the corresponding voice message, thus more accurately determining the user's real-time emotion. The initial complaint classification model is used to analyze the user's historical profile data (representing the user's historical behavior) to obtain a complaint escalation prediction result. The initial prediction model is used to comprehensively analyze the emotion recognition result, sentiment tendency result, and complaint escalation prediction result to determine the complaint escalation score. Since the emotion recognition result and sentiment tendency result are obtained from real-time user behavior analysis (such as historical complaint voice data), and the complaint escalation prediction result is obtained from historical user behavior analysis (such as historical profile data), the model simultaneously considers the analysis of both real-time and historical user behavior, improving the accuracy of the obtained complaint escalation score.

[0057] In the model training method of this application, the training data set includes historical user profile data, historical complaint voice data corresponding to the historical user profile data, emotion tags corresponding to the historical complaint voice data, sentiment tendency tags corresponding to the historical complaint voice data, and complaint escalation tags corresponding to the historical user profile data. The initial model includes an initial emotion recognition model, an initial sentiment analysis model, an initial complaint classification model, and an initial prediction model. Multiple training data sets are used to train the initial model to obtain the target model. The target model can output emotion recognition results and sentiment tendency results based on the user's complaint voice data, and can output complaint escalation prediction results based on the user's user profile data. Since the emotion recognition results and sentiment tendency results are derived from analyzing the user's real-time behavior, and the escalation prediction results are derived from analyzing the user's historical behavior, the target model can simultaneously consider the impact of the user's real-time behavior and historical behavior on whether the user will escalate the complaint, thereby improving the accuracy of the obtained complaint escalation score, which is equivalent to improving the prediction accuracy of whether the user will escalate the complaint.

[0058] Optionally, training the initial model based on the plurality of training data sets to obtain the target model includes:

[0059] The historical complaint voice data in the plurality of training data groups are preprocessed to obtain a first voice data set. The first voice data set includes all historical complaint voice data after preprocessing in the plurality of training data groups. The voice data preprocessing includes at least one of the following: noise reduction, frame segmentation, windowing, and length alignment.

[0060] Each historical complaint voice data after preprocessing in the first voice data set is converted into text to obtain a first text feature set, which includes all text features after text conversion in the first voice data set.

[0061] The historical user profile data in the plurality of training data groups are preprocessed to obtain a first profile data set. The first profile data set includes all historical user profile data after the profile data preprocessing in the plurality of training data groups. The profile data preprocessing includes at least one of the following: handling missing values ​​of numerical features, handling missing values ​​of categorical features, handling extreme missing values, and normalization.

[0062] Based on the emotion tags in the first speech data set, the first text feature set, and the plurality of training data sets, the initial emotion recognition model is trained to obtain a target emotion recognition model; and based on the sentiment tendency tags in the first text feature set and the plurality of training data sets, the initial sentiment analysis model is trained to obtain a target sentiment analysis model; and based on the complaint escalation tags in the first profile data set and the plurality of training data sets, the initial complaint classification model is trained to obtain a target complaint classification model; wherein, the target model includes the target emotion recognition model, the target sentiment analysis model, and the target complaint classification model.

[0063] In this embodiment, historical complaint voice data is preprocessed to obtain a first voice data set and a first text feature set. The first voice data set and the first text feature set can be used to train the initial emotion recognition model, and the first text feature set can be used to train the initial sentiment analysis model. Furthermore, this embodiment also preprocesses historical user profile data to obtain a first profile data set, which can be used to train the initial complaint classification model.

[0064] Voice data preprocessing may include:

[0065] Noise reduction: Wavelet denoising is used to reduce the impact of environmental noise on audio quality;

[0066] Framing / Windowing: Divide long speech into short frames (e.g., 20ms / frame) and add a Hanning window to reduce spectral leakage;

[0067] Length alignment: If audio lengths are inconsistent, the input length is unified by truncation / zero padding;

[0068] Speech feature extraction: Mel-frequency cepstral coefficients (MFCCs) and F0 (reflecting pitch and loudness) are extracted to generate a temporal feature sequence; a pre-trained acoustic model (Wav2Vec2) is used to extract fixed-dimensional embedding vectors (e.g., [1, 768]) to capture high-level prosodic patterns. The structure of the speech feature extraction model network is as follows: Figure 3 As shown.

[0069] The text conversion can be:

[0070] The recording was transcribed into a draft text using a speech recognition tool (Whisper), followed by manual revisions and speaker annotations. This text was then matched against the previously manually annotated sentiment and emotion. The draft text was processed through methods such as... Figure 4 The text feature extraction model network shown obtains text features through processing.

[0071] Image data preprocessing may include:

[0072] Handling missing values ​​for numerical features: Fill with the median (the median is more robust if the majority of user complaints are 0).

[0073] Handling missing values ​​for categorical features: fill in with the mode (e.g., "Gold" is the most common level), or add a new "Unknown" category;

[0074] Extreme missing features (missing rate of a certain feature > 30%): directly delete the feature to avoid noise interference.

[0075] Normalization process:

[0076] Unordered categorical feature encoding and normalization (e.g., region, membership level): use One-Hot encoding;

[0077] Ordinal categorical feature encoding and normalization (e.g., age stratification 20-30 / 30-40): use LabelEncoding;

[0078] Numerical feature normalization: For features with large differences between data clusters, use Min-Max normalization (scaled to [0,1]) to eliminate the influence of the unit on DNN training.

[0079] In this embodiment, the historical complaint voice data and historical user profile data are preprocessed to improve data quality, thereby improving the model training effect.

[0080] Optionally, training the initial emotion recognition model based on the first speech data set, the first text feature set, and the emotion labels in the plurality of training data sets to obtain the target emotion recognition model includes:

[0081] The text features in the first text feature set are preprocessed to obtain the second text feature data set. The text feature preprocessing includes at least one of the following: text cleaning and semantic encoding.

[0082] A first enhancement process is performed on at least one preprocessed historical complaint voice data in the first voice data set to obtain at least one enhanced voice data that corresponds one-to-one with the at least one preprocessed historical complaint voice data. The first enhancement process includes at least one of the following: adding background noise and fundamental frequency transformation.

[0083] The at least one enhanced speech data is added to the first speech data set to obtain the second speech data set;

[0084] A second enhancement process is performed on at least one first text feature in the second text feature set to obtain at least one first enhanced text feature that corresponds one-to-one with the at least one first text feature. The second enhancement process includes at least one of the following: synonym replacement, random insertion or deletion of word elements;

[0085] The at least one first enhanced text feature is added to the second text feature set to obtain a third text feature set;

[0086] Based on the second speech data set, the third text feature set, and the emotion labels in the multiple training data sets, the initial emotion recognition model is trained to obtain the target emotion recognition model.

[0087] In this embodiment, the text feature preprocessing may include:

[0088] Text cleaning: Filtering out modal particles and repeated words, and correcting ASR errors;

[0089] Semantic encoding: The text is encoded using the pre-trained language model BERT, and the output sentence vector is taken as the [CLS] token, with a shape of [1, 768].

[0090] The first enhancement process is speech data enhancement: adding background noise and pitch transformation (simulating pitch changes of different emotions).

[0091] The second enhancement process is text data enhancement: synonym replacement, random insertion / deletion (enhancing semantic robustness).

[0092] By performing the first and second enhancement processes, the amount of data in the second speech data set and the third text feature set can be increased, thereby increasing the amount of training data and improving the training effect of the initial emotion recognition model.

[0093] It should be noted that the emotion label corresponding to the first enhanced speech data is the same as the emotion label corresponding to the original speech data, and the emotion label corresponding to the first enhanced text feature is the same as the emotion label corresponding to the speech data of the original text feature.

[0094] The second speech data set and the third text feature set can be split into training, validation, and test sets in a 7:2:1 ratio to ensure a balanced distribution of emotion labels. The initial emotion recognition model is trained using the split training, validation, and test sets. The training method is similar to that of general models using training, validation, and test sets, and will not be described in detail in this application.

[0095] Optionally, training the initial sentiment analysis model based on the first text feature set and the sentiment tendency labels in the plurality of training data sets to obtain the target sentiment analysis model includes:

[0096] The text features in the first text feature set are subjected to a first preprocessing to obtain a fourth text feature set. The first preprocessing includes at least one of the following: text clarification, dialogue structuring, and format conversion.

[0097] A third enhancement process is performed on at least one first preprocessed first text feature in the fourth text feature set to obtain at least one second enhanced text feature that corresponds one-to-one with the at least one first preprocessed first text feature. The third enhancement process includes at least one of the following: synonym sentiment word replacement, back translation enhancement, and generative enhancement.

[0098] The at least one second enhanced text feature is added to the fourth text feature set to obtain the fifth text feature set;

[0099] Based on the fifth text feature set and the sentiment tendency labels in the multiple training data sets, the initial sentiment analysis model is trained to obtain the target sentiment analysis model.

[0100] In this embodiment, the first preprocessing may include:

[0101] Text cleaning: Filtering irrelevant information: removing timestamps, system prompts, and repetitive verbal tics.

[0102] Structured Dialogue: Breaking Down Multi-Turn Dialogues: Dividing dialogues into rounds of "user speaks → operator responds," focusing on user text (the core analysis object).

[0103] Format conversion: Convert text into a large model-compatible format (JSON).

[0104] The third enhancement process can optimize data distribution, enrich sample diversity, and improve the generalization ability of large models. The third enhancement process may include:

[0105] Data augmentation: Replace sentiment words in user text with NLP tools (e.g., "anger" → "annoyance") while preserving semantic sentiment;

[0106] Back-translation enhancement: Translate the text into English and then back into Chinese using the Tongyi Qianwen method to generate samples with similar meanings but different expressions;

[0107] Generative enhancement: Use Tongyi Qianwen to perform "Generate similar expressions with the same sentiment based on the following text: [original text]", expand the niche sentiment samples (such as the diverse expressions of "dissatisfaction"), and make each sentiment label maintain an approximately proportional distribution in each subset.

[0108] The sentiment label corresponding to the second enhanced text feature is the same as the sentiment label corresponding to the original first text feature.

[0109] The fifth text feature set can be split into a training set, a validation set, and a test set in a 7:2:1 ratio, and used to train the initial sentiment analysis model.

[0110] By employing the first preprocessing and third enhancement processes described above, the amount and quality of data in the fifth text feature set can be increased, thereby improving the training effect of the initial sentiment analysis model.

[0111] In an optional implementation, the target sentiment analysis model can directly adopt an existing large-scale dialogue model. The method for performing sentiment analysis using an existing large-scale dialogue model can be as follows:

[0112] PI Invocation: Guides large model inference using a Prompt template. An example of a Prompt is shown below.

[0113] Task: Analyze the sentiment of user conversations;

[0114] Input text: "I've waited three days and no one has dealt with it. Do you even have a sense of responsibility?"

[0115] Output requirements: Output emotion label (optional: anger, dissatisfaction, neutral, satisfaction);

[0116] Input layer: Organizes preprocessed text (such as user dialogue + operator summary) in Prompt format;

[0117] Inference layer: Calls the Tongyi Thousand Questions API to output sentiment bias and confidence level;

[0118] Post-processing: Parsing the output of a large model. An example of a large model output is shown below:

[0119] user_text (user text): "I've been waiting for three days and no one has dealt with it. Do you even have any sense of responsibility?"

[0120] emotion_label: "anger";

[0121] Confidence level: 0.92 (i.e., 92% confidence level);

[0122] Performance tuning: Optimize Prompt and add Few-Shot examples (showing large models examples of "text → label") to improve accuracy;

[0123] Fine-tune the strategy by incorporating domain prompts (such as "sentiment analysis in customer service complaint scenarios") to guide the model to focus on business needs.

[0124] In this embodiment, the existing large dialogue model is directly used, which can save the initial sentiment analysis model training process and improve the execution efficiency of the method.

[0125] Optionally, the step of training the initial complaint classification model based on the complaint escalation tags in the first profile data set and the plurality of training data sets to obtain the target complaint classification model includes:

[0126] Virtual sample generation is performed on at least one first profile data in the first profile data set to obtain at least one first enhanced profile data, wherein the complaint escalation tag corresponding to the first profile data is a first tag, and the first tag is used to indicate that the user has made an escalation complaint.

[0127] Add the at least one first enhanced portrait data to the first portrait data set to obtain a second portrait data set;

[0128] At least one second profile data is deleted from the second profile data set to obtain a third profile data set, wherein the complaint escalation label corresponding to the second profile data is the second label, and the first label is used to indicate that the user has not escalated the complaint;

[0129] Based on the third profile dataset and the complaint escalation tags in multiple training data sets, the initial complaint classification model is trained to obtain the target complaint classification model.

[0130] In this embodiment, since the number of users who escalated their complaints is usually far less than the number of users who did not escalate their complaints in multiple training data sets, it is necessary to balance the number of samples.

[0131] The sample size balancing involves oversampling the first profile data and undersampling the second profile data.

[0132] Oversampling:

[0133] Virtual complaint samples can be generated from the first profile data set whose complaint escalation label is the first label, to obtain the first enhanced profile data. The virtual complaint samples can be generated by interpolation based on similar samples.

[0134] Undersampling:

[0135] Delete at least one second portrait data from the second portrait data set to reduce the number of second portrait data.

[0136] It should be noted that the complaint escalation label corresponding to the first enhanced profile data is the first label.

[0137] The third profile dataset can be split into training, validation, and test sets in a 7:2:1 ratio to train the initial complaint classification model. During training, higher loss weights can be assigned to the data corresponding to the first label (e.g., the weight of the sample with y=1 is 5, and the weight of y=0 is 1), forcing the model to pay more attention to the minority class data, making the target complaint classification model more accurate.

[0138] In an optional implementation, the initial complaint classification model includes:

[0139] Input layer: Dimension = Total number of preprocessed features (e.g., user tags + complaint history + auxiliary features totaling 102 → input layer dimension 102);

[0140] Hidden layers (multi-layer nonlinear transformation): 3 layers; the number of neurons in each layer decreases by 0.5 times that of the previous layer (128 in the first layer, 64 in the second layer, and 32 in the third layer).

[0141] Activation function: ReLU (alleviates gradient vanishing and accelerates training);

[0142] Regularization: Add Dropout (probability 0.2) after each layer to prevent overfitting.

[0143] Output layer: 1 neuron (binary classification task);

[0144] Activation function: Sigmoid (outputs the probability of a complaint as 0 to 1).

[0145] The initial complaint classification model training process includes:

[0146] A[Load training / validation set] --> B[Forward propagation: Input features → DNN → Output complaint probability (complaint escalation prediction result)];

[0147] B --> C [Calculate the loss: cross-entropy of the true label (complaint escalation label) vs. the predicted probability (complaint escalation prediction result)];

[0148] C --> D [Backpropagation: Gradient descent updates the parameters of the entire network];

[0149] Does the validation set loss decrease when D --> E?

[0150] E -- Yes --> F [Save the current optimal model];

[0151] E -- No --> G [Trigger early stop / adjust learning rate];

[0152] G --> A.

[0153] After the model training is completed, the complaint prediction results need to be output in the production environment for new user data (real-time / offline data that did not participate in the training). The process revolves around "data alignment → model loading → inference calculation → result application".

[0154] Inference data preprocessing (new data needs to replicate the preprocessing process used during training to ensure that the input distribution is consistent with the training data, thus avoiding inference bias):

[0155] Feature extraction: Retrieve user tags, complaint history, and other data from business systems;

[0156] Missing values / encoding / normalization: Missing values: handled using imputation rules from the training phase;

[0157] Category encoding: Call the One-Hot mapping table saved during training;

[0158] Numerical normalization: processed using the Min / Max statistics from the training set;

[0159] Feature dimension verification: Ensure that the number and order of features in the new data are completely consistent with those in the training input;

[0160] Model loading and inference engine selection:

[0161] Export the model using TensorFlow as a SavedModel format;

[0162] Save the preprocessing parameters and encapsulate them into an integrated service of "model + preprocessing logic";

[0163] Reasoning, calculation, and result output:

[0164] Forward propagation: Perform forward computation of the DNN's "input layer → hidden layer → output layer" to obtain the complaint probability;

[0165] Result analysis: Set a threshold (e.g., 0.5), a probability ≥ 0.5 is considered high risk for complaints, otherwise it is considered low risk; output the feature contribution to explain which features lead to high risk;

[0166] Output format: Returns a structured JSON result (including user ID, complaint probability, and risk label) for use by customer service systems, operations dashboards, and other business applications.

[0167] The performance of the target complaint classification model during the inference stage directly affects business value and requires targeted optimization.

[0168] Inference efficiency tuning (reducing latency and increasing throughput):

[0169] Model structure compression:

[0170] Quantization: Convert the model weights from FP32 to INT8 to reduce computation and memory usage (accuracy loss needs to be verified to be <5%).

[0171] Pruning: Guided by L1 / L2 regularization, remove "small-contribution connections / neurons" in the DNN to reduce the model size;

[0172] Multi-Batch Inference: Merging multiple user requests into a single batch input model;

[0173] Asynchronous inference: Use multithreading / asynchronous I / O to separate "data reading" from "model calculation" and hide I / O time consumption.

[0174] Accelerating the Data Pipeline:

[0175] Preprocessing vectorization: Replace loops with NumPy, or use the built-in preprocessing module of TensorRT Inference Server to accelerate feature transformation;

[0176] Hot data caching: For frequently queried features, cache them in in-memory databases such as Redis to reduce database I / O time.

[0177] Monitoring and correction:

[0178] Concept drift detection: Use the PSI metric to monitor and adjust the feature distribution differences between new and training data;

[0179] Incremental training: New complaint data is periodically added to the training set, and online learning is achieved using the Flink + TensorFlow framework to dynamically reduce the error of a single model.

[0180] The above methods can improve the accuracy of the results output by the target complaint classification model, thereby improving the accuracy of predictions on whether a user's complaint will escalate.

[0181] Optionally, the initial emotion recognition model includes a cross-attention interaction layer, a multimodal fusion layer, and an emotion classification layer;

[0182] The cross-attention interaction layer is used to: enhance text features using speech features to obtain text-enhanced features, and enhance speech features using the text features to obtain speech-enhanced features, wherein the text features are features obtained by text conversion of complaint speech data input into the initial emotion recognition model, and the speech features are features obtained by acoustic feature extraction of complaint speech data input into the initial emotion recognition model.

[0183] The multimodal fusion layer is used to: non-linearly integrate the text enhancement features and the speech enhancement features to obtain fused features;

[0184] The emotion classification layer is used to: perform emotion recognition on the fused features and output the emotion recognition result.

[0185] In this embodiment, as Figure 5 As shown, the initial emotion recognition model includes a cross-attention interaction layer, a multimodal fusion layer, and an emotion classification layer.

[0186] Among them, the cross-attention interaction layer is used for "speech-to-text enhancement" and "text-to-speech enhancement" as well as multi-head cross-attention:

[0187] Speech-to-text enhancement: Using speech features as the query and text features as the key and value, we calculate attention weights (the correlation between speech segments and different parts of the text), sum the text features by weight, and generate "text enhancement features from the perspective of speech".

[0188] Text-to-speech enhancement: Using text features as the query and speech features as the key and value, we calculate attention weights (the degree of correlation between each part of the text and the speech segment), sum the speech features by weight, and generate "speech enhancement features from the perspective of text".

[0189] Multi-Head Cross-Attention: To capture multi-dimensional modal interactions, features are split into 8 subspaces, cross-attention is computed independently in each subspace, and the results of the subspaces are concatenated to enhance the richness of the interactions.

[0190] The multimodal fusion layer is used to non-linearly integrate the attention-enhanced bimodal features (speech enhancement features + text enhancement features) to generate a unified sentiment representation.

[0191] Feature concatenation + MLP: Directly concatenate speech and text enhancement features → feed into a multilayer perceptron (MLP) for nonlinear transformation, add residual connections to prevent gradient vanishing, and add dropout regularization to prevent overfitting.

[0192] Hierarchical fusion (for temporal features): If the speech / text features are temporal sequences (such as segmented features of long dialogues), first fuse the local temporal sub-features (such as using LSTM to capture emotional changes within short segments), and then fuse the global sequence (such as using Transformer to capture emotional trends across segments) to simulate the "local-global" emotional expression logic.

[0193] The emotion classification layer is used to: feed the fused features into a fully connected layer (FC) + Softmax, and output the emotion recognition result, which can be the probability distribution of emotion categories (such as happiness, sadness, anger, neutrality, etc.).

[0194] In this embodiment, the initial emotion recognition model can enhance and fuse text features and speech features through the above-mentioned hierarchical network, so that the fused features can more accurately reflect the user's emotions, thereby improving the accuracy of emotion recognition results.

[0195] like Figure 2 As shown, the modeling process for the initial model described above can be summarized as follows:

[0196] 1. Problem Definition

[0197] Model objective: To predict the likelihood that a user will escalate their complaint in the near future.

[0198] 2. Data Collection

[0199] Collect data related to user behavior, satisfaction, complaint history, complaint voice recordings, and voice-text recordings, including but not limited to:

[0200] User basic information (age, gender, geographical location, etc.);

[0201] User interaction records (mobile product subscription history, service usage, etc.);

[0202] Historical complaint data (complaint time, reason for complaint, handling result, etc.);

[0203] User satisfaction survey results;

[0204] User feedback on social media and forums;

[0205] Incoming call content (text and voice recordings, customer service communication records, call voice recordings, etc.).

[0206] 3. Feature Selection

[0207] Methods such as statistical testing, information gain, feature importance scoring, feature data distribution analysis, and feature correlation analysis are used to select the features most useful for predicting complaints. Dimensionality reduction techniques such as Principal Component Analysis (PCA) are applied to reduce the number of features.

[0208] 4. Data Preprocessing

[0209] Numerical data: missing value imputation, excessively sparse data removal, outlier detection and removal, data binning, numerical test analysis, etc.

[0210] Categorical data: Categorical noise data removal, categorical mapping, sparse categorical filling, etc.

[0211] Text data preprocessing: invalid character removal, abnormal character filtering, long text truncation, text role identification, etc.

[0212] Voice data preprocessing: noise removal, voice slicing, voice role identification, etc.

[0213] Document content parsing: parsing work order texts, parsing user feedback content, etc.

[0214] Training sample segmentation: Users who have filed a narrow-sense complaint and then escalated it to a complaint about the same type of problem within the past three months; and users who have filed a narrow-sense complaint but have not escalated it to a complaint about the same type of problem within the past three months.

[0215] Training sample annotation: The positive and negative sample annotation of speech segments adopts a semi-automatic annotation method. Specifically, the locally deployed Whisper model is used to convert the segmented speech data into text segments. Then, the understanding ability of the large language model (Tongyi Qianwen) is used to perform coarse-grained annotation of the text segments of the speech. Then, further fine-grained annotation is performed by manual calibration, so as to obtain the training samples of the speech emotion recognition model with positive emotion speech segments labeled as S and negative emotion speech segments labeled as D.

[0216] 5. Feature Engineering

[0217] Numerical data: numerical feature processing methods such as uniform quantization and feature standardization.

[0218] Categorical data: Feature processing methods such as one-hot, multi-hot, and category mapping.

[0219] Text-based data: text semantic embedding technology, n-gram technology, etc.

[0220] Speech data: techniques and methods such as Mel spectrum, MFCC feature extraction, and wav2vec2 feature extraction.

[0221] 6. Establish a model for the complaint prediction device.

[0222] The prediction process of the voice emotion recognition model is as follows: First, the complaint voice is segmented into pieces, then the segment features (MFCC features and wav2vec2 features) are extracted, and finally the emotion recognition result is obtained by reasoning through the emotion recognition model (CA-MSER).

[0223] The text sentiment analysis model prediction process is as follows: First, the input complaint speech is converted into speech text using the Whisper speech recognition model. Then, it is cleaned together with the parsed text of the complaint work order and feedback. Finally, the sentiment tendency is determined by the Qianwen big data model, and a sentiment tendency value is given.

[0224] The inference process of the complaint escalation prediction model is as follows: First, the user profile wide table data (excluding voice) is accessed, and then data preprocessing and feature engineering are performed. Finally, the trained complaint prediction model is used to give the predicted value of complaint escalation.

[0225] The normalization operator integrates the prediction results: This part integrates the outputs of the speech emotion recognition model, the text sentiment analysis model, and the complaint escalation prediction model, and normalizes and weights them to obtain the complaint escalation prediction probability. In specific experiments, the best complaint prediction effect was achieved when the weighting coefficients were set to the following formula using the grid search parameter iteration method: Score (complaint escalation prediction device score) = 0.53res (speech emotion recognition model) + 0.35res (text sentiment analysis model) + 0.12 (complaint prediction model).

[0226] Model Training: The complaint escalation prediction model is trained using historical user profile data, with complaining users as positive samples and non-complaining users as negative samples; the speech emotion recognition model is trained using labeled speech segment samples; the big oracle model does not require training; the normalized weighted operator does not require training.

[0227] 7. Model Evaluation

[0228] Split the dataset: Divide the data into training set, validation set and test set.

[0229] Evaluation metrics: The model performance is evaluated using metrics such as accuracy, precision, recall, F1 score, and ROC curve.

[0230] Cross-validation: Use cross-validation to evaluate the stability and generalization ability of the model.

[0231] 8. Model Optimization

[0232] The optimal model parameters are found using methods such as grid search and random search. The prediction results of multiple models are combined, and a linear weighted algorithm is used to comprehensively evaluate the results of each model to improve prediction accuracy.

[0233] 9. Deployment and Monitoring

[0234] Deploy the trained model to the production environment to predict potential complainants in real-time or in batches. Performance monitoring: Continuously monitor the model's performance to ensure its predictive accuracy and stability.

[0235] See Figure 6 , Figure 6 This is a flowchart of the model training method provided in the embodiments of this application, such as... Figure 6 As shown, it includes the following steps:

[0236] Step 601: Input the complaint voice data of the target user and the user profile data of the target user into the target model to obtain the emotion recognition result, sentiment tendency result, complaint escalation prediction result, and complaint escalation score output by the target model. The target model is a model trained based on the above-described model training method. The target model includes a target emotion recognition model, a target sentiment analysis model, a target complaint classification model, and a target prediction model. The target emotion recognition model is used to: receive the complaint voice data, perform emotion recognition on the complaint voice data, and output the emotion recognition result. The target sentiment analysis model is used to: receive the complaint voice data, perform sentiment tendency analysis on the complaint voice data, and output the sentiment tendency result. The target complaint classification model is used to: receive the user profile data, predict whether the user will escalate their complaint based on the user profile data, and output the complaint escalation prediction result. The target prediction model is used to: receive the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determine the complaint escalation score of the target user based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation prediction score is used to indicate the probability that the target user will escalate their complaint.

[0237] In this embodiment, the complaint voice data of the target user and the user profile data of the target user are input into the target model. The target model analyzes the real-time behavior data of the user, namely the complaint voice data, and analyzes the historical behavior data of the user, namely the user profile data. This can improve the accuracy of the obtained complaint escalation score, which is equivalent to improving the prediction accuracy of whether the user will escalate the complaint.

[0238] Optionally, the emotion recognition result includes an emotion recognition score, the sentiment tendency result includes a sentiment tendency score, and the complaint escalation prediction result includes a complaint escalation prediction score.

[0239] The step of determining the complaint escalation score of the target user based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result includes:

[0240] The emotion recognition score, the sentiment tendency score, and the complaint escalation prediction score are normalized to obtain a first normalized value corresponding to the emotion recognition score, a second normalized value corresponding to the sentiment tendency score, and a third normalized value corresponding to the complaint escalation prediction score.

[0241] Using preset weight information, the first normalized value, the second normalized value, and the third normalized value are weighted and summed to obtain the complaint escalation score. The preset weight information is used to indicate: a first weight corresponding to the emotion recognition score, a second weight corresponding to the emotion tendency score, and a third weight corresponding to the complaint escalation prediction score. The sum of the first weight, the second weight, and the third weight is 1.

[0242] In this embodiment, the emotion recognition score can be denoted as res_audio, the sentiment tendency score can be denoted as res_text, and the complaint escalation prediction score can be denoted as res_dnn.

[0243] Because the output values ​​of the three models have different ranges and units, direct weighting would result in an excessively large range. Therefore, the outputs of each model are first normalized to a uniform interval (e.g., 0 - 1). The Min-Max normalization method is commonly used.

[0244] Normalize res_audio: Substitute the minimum value min_audio and the maximum value max_audio into the formula to obtain res_audio_norm (the first normalized value);

[0245] Normalize res_text: Similarly, calculate the min_text and max_text output by the Qianwen model to obtain res_text_norm (the second normalized value).

[0246] Normalize res_dnn: Calculate the min_dnn and max_dnn outputs of the DNN model to obtain res_dnn_norm (the third normalization value).

[0247] The preset weight information is determined through a grid search parameter iteration method. When the weights indicated by the preset weight information are set according to the following formula, the best complaint prediction effect is achieved:

[0248] Score (complaint escalation score) = 0.53*res_audio_norm + 0.35*res_text_norm +0.12*res_dnn_norm.

[0249] The complaint escalation score directly outputs the probability that a complainant will escalate their complaint, and the business system can take action based on this probability:

[0250] If the score is ≥ 0.7 (the threshold can be customized), it is judged as "high upgrade risk";

[0251] If 0.3 ≤ Score < 0.7, it is judged as "medium risk";

[0252] If the score is less than 0.3, it is considered "low risk".

[0253] After identifying risks, staff can be guided to intervene in real time, such as reassuring users and transforming "post-event analysis" into "in-event intervention." Proactively responding during the golden window of opportunity when customer emotions / demands escalate can improve user satisfaction.

[0254] The advantages of the model training method and the escalation complaint behavior prediction method of this application compared with the prior art are explained as follows:

[0255] Breaking away from the limitations of traditional methods that rely solely on single structured data, this approach comprehensively integrates multimodal data, including recorded speech (unstructured audio), transcribed text, complaint work order text, and satisfaction feedback text. It leverages acoustic sentiment analysis to capture the prosodic features of angry intonations in speech and semantic parsing to uncover the logical chains of implicit demands within work orders. From the emotional texture of the voice to the semantic context of the text, it constructs a three-dimensional information network—ensuring that easily overlooked emotional cues in complaint scenarios (such as the customer's restrained dissatisfaction or vague demands in the work order) can be accurately captured.

[0256] Unlike the fragmented "single-point service" approach of traditional methods, this application achieves full lifecycle coverage from before to after the complaint process:

[0257] Before a complaint is filed: Use multimodal risk identification to locate users with a high tendency to file complaints and intervene in advance to block them;

[0258] During a complaint: Generate guidance suggestions based on real-time emotion / semantic analysis (e.g., automatically trigger empathetic dialogue for angry customers);

[0259] Following a complaint: Analyze the textual attribution of satisfaction ratings to identify deficiencies (e.g., if there is a concentration of logistics complaints, promote warehouse optimization).

[0260] From "risk prediction" to "operational intervention" and then to "root cause governance", a complete closed loop of business operations is formed, allowing the value of data to run through the entire complaint management process.

[0261] Traditional feature engineering methods remain at the level of "behavioral statistics and numerical similarity," while this application achieves a leap in feature dimensions through multimodal intelligent analysis:

[0262] On the text side: NLP emotion recognition outputs emotional polarities such as "anger / sadness" and semantic intents such as "refund / apology"; the large model analyzes contextual logic such as "previous promises not being fulfilled" to uncover the core motivations for complaints;

[0263] On the group segmentation side: Incorporating soft features such as "high anger groups" and "logistics complaint groups" breaks the limitations of static labels such as "consumption amount and region", and achieves accurate differentiation between "rational demand groups (requiring efficient solutions)" and "emotional venting groups (requiring emotional comfort)", upgrading the group segmentation logic from "numerical similarity" to "business semantic similarity".

[0264] Addressing the core demand in complaint scenarios that "real-time performance determines the value of intervention," this application establishes a streaming processing + large-scale model real-time inference architecture:

[0265] For "ongoing complaint recordings", ASR real-time transcription + sentiment analysis outputs sentiment tags in seconds (e.g., when a customer's emotions suddenly rise during the call, the system automatically issues a warning and triggers the "transfer to senior customer service" instruction).

[0266] For newly submitted work orders, the big data model analyzes the semantics in real time and pushes the processing priority (such as expedited processing of work orders containing the keyword "fraud").

[0267] Transform "post-event analysis" into "in-event intervention," proactively responding during the golden window of opportunity when customer emotions / demands are brewing.

[0268] Traditional methods are limited by the "traditional supervised / unsupervised learning" paradigm, proving ineffective for speech, long texts, and open-domain semantics. This application achieves a breakthrough by combining multimodal technologies:

[0269] Audio modality: ASR (speech-to-text) + acoustic sentiment analysis, solving the problem of understanding "unstructured recordings";

[0270] Text Modality: NLP emotion recognition (such as BERT - Emotion) + large models (such as Tongyi Qianwen) to overcome the pain points of parsing "colloquial complaints in work orders and semantic ambiguity in long texts".

[0271] The technology stack is naturally adapted to complaint scenarios that are "multimodal, unstructured, and open-domain," enabling "emotional fluctuations in voice and implicit demands in text" to be intelligently understood.

[0272] See Figure 7 , Figure 7 This is a structural diagram of an escalation complaint behavior prediction device provided in an embodiment of this application. Figure 7 As shown, the device 700 includes:

[0273] The acquisition module 701 is used to acquire multiple training data sets, the training data sets including historical user profile data, historical complaint voice data corresponding to the historical user profile data, emotion tags corresponding to the historical complaint voice data, sentiment tendency tags corresponding to the historical complaint voice data, and complaint escalation tags corresponding to the historical user profile data. The complaint escalation tags are used to indicate whether the user corresponding to the historical user profile data should escalate the complaint when the user sends the voice corresponding to the historical complaint voice data.

[0274] Training module 702 is used to train an initial model based on the multiple training data sets to obtain a target model. The initial model includes an initial emotion recognition model, an initial sentiment analysis model, an initial complaint classification model, and an initial prediction model. The initial emotion recognition model is used to: receive the historical complaint voice data, perform emotion recognition on the historical complaint voice data, and output an emotion recognition result. The initial sentiment analysis model is used to: receive the historical complaint voice data, perform sentiment tendency analysis on the historical complaint voice data, and output a sentiment tendency result. The initial complaint classification model is used to: receive the historical user profile data, predict whether a user will escalate their complaint based on the historical user profile data, and output a complaint escalation prediction result. The initial prediction model is used to: receive the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determine a complaint escalation score based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation score indicates the probability of the corresponding user escalating their complaint.

[0275] The device 700 further includes:

[0276] Prediction module 703 is used to input the complaint voice data of the target user and the user profile data of the target user into the target model, and obtain the emotion recognition result, sentiment tendency result, complaint escalation prediction result, and complaint escalation score output by the target model. The target model is a model trained based on the model training method described above, and includes a target emotion recognition model, a target sentiment analysis model, a target complaint classification model, and a target prediction model. The target emotion recognition model is used to: receive the complaint voice data, perform emotion recognition on the complaint voice data, and output the emotion recognition result; the target sentiment analysis model is used to... The system is configured to: receive the complaint voice data, perform sentiment analysis on the complaint voice data, and output the sentiment result; the target complaint classification is configured to: receive the user profile data, predict whether the user will escalate the complaint based on the user profile data, and output the complaint escalation prediction result; the target prediction model is configured to: receive the emotion recognition result, the sentiment result, and the complaint escalation prediction result, and determine the complaint escalation score of the target user based on the emotion recognition result, the sentiment result, and the complaint escalation prediction result, wherein the complaint escalation prediction score is used to indicate the probability of the target user escalating the complaint.

[0277] Optionally, the training module 702 is also used for:

[0278] The historical complaint voice data in the plurality of training data groups are preprocessed to obtain a first voice data set. The first voice data set includes all historical complaint voice data after preprocessing in the plurality of training data groups. The voice data preprocessing includes at least one of the following: noise reduction, frame segmentation, windowing, and length alignment.

[0279] Each historical complaint voice data after preprocessing in the first voice data set is converted into text to obtain a first text feature set, which includes all text features after text conversion in the first voice data set.

[0280] The historical user profile data in the plurality of training data groups are preprocessed to obtain a first profile data set. The first profile data set includes all historical user profile data after the profile data preprocessing in the plurality of training data groups. The profile data preprocessing includes at least one of the following: handling missing values ​​of numerical features, handling missing values ​​of categorical features, handling extreme missing values, and normalization.

[0281] Based on the emotion tags in the first speech data set, the first text feature set, and the plurality of training data sets, the initial emotion recognition model is trained to obtain a target emotion recognition model; and based on the sentiment tendency tags in the first text feature set and the plurality of training data sets, the initial sentiment analysis model is trained to obtain a target sentiment analysis model; and based on the complaint escalation tags in the first profile data set and the plurality of training data sets, the initial complaint classification model is trained to obtain a target complaint classification model; wherein, the target model includes the target emotion recognition model, the target sentiment analysis model, and the target complaint classification model.

[0282] Optionally, the training module 702 is also used for:

[0283] Text features in the first text feature set are preprocessed to obtain a second text feature data set. The text feature preprocessing includes at least one of the following: text cleaning and semantic encoding. At least one preprocessed historical complaint voice data in the first voice data set is subjected to a first enhancement process to obtain at least one enhanced voice data corresponding one-to-one with the preprocessed historical complaint voice data. The first enhancement process includes at least one of the following: adding background noise and fundamental frequency transformation. The at least one enhanced voice data is added to the first voice data set to obtain a second voice data set. At least one first text feature in the second text feature set is subjected to a second enhancement process to obtain at least one first enhanced text feature corresponding one-to-one with the at least one first text feature. The second enhancement process includes at least one of the following: synonym replacement and random insertion or deletion of lexical units. The at least one first enhanced text feature is added to the second text feature set to obtain a third text feature set. Based on the second voice data set, the third text feature set, and the emotion labels in the plurality of training data sets, the initial emotion recognition model is trained to obtain the target emotion recognition model.

[0284] And / or, perform a first preprocessing on the text features in the first text feature set to obtain a fourth text feature set, wherein the first preprocessing includes at least one of the following: text clarity, dialogue structuring, and format conversion; perform a third enhancement processing on at least one of the first preprocessed first text features in the fourth text feature set to obtain at least one second enhanced text feature corresponding one-to-one with the at least one first preprocessed first text feature, wherein the third enhancement processing includes at least one of the following: synonym sentiment word replacement, back-translation enhancement, and generative enhancement; add the at least one second enhanced text feature to the fourth text feature set to obtain a fifth text feature set; and train the initial sentiment analysis model based on the fifth text feature set and the sentiment tendency labels in the plurality of training data sets to obtain the target sentiment analysis model;

[0285] And / or, at least one first profile data in the first profile data set is used to generate virtual samples to obtain at least one first enhanced profile data, wherein the complaint escalation label corresponding to the first profile data is a first label, which is used to indicate that the user has escalated the complaint; the at least one first enhanced profile data is added to the first profile data set to obtain a second profile data set; at least one second profile data is deleted from the second profile data set to obtain a third profile data set, wherein the complaint escalation label corresponding to the second profile data is a second label, and the first label is used to indicate that the user has not escalated the complaint; based on the third profile data set and the complaint escalation labels in multiple training data sets, the initial complaint classification model is trained to obtain the target complaint classification model.

[0286] Optionally, the initial emotion recognition model includes a cross-attention interaction layer, a multimodal fusion layer, and an emotion classification layer;

[0287] The cross-attention interaction layer is used to: enhance text features using speech features to obtain text-enhanced features, and enhance speech features using the text features to obtain speech-enhanced features, wherein the text features are features obtained by text conversion of complaint speech data input into the initial emotion recognition model, and the speech features are features obtained by acoustic feature extraction of complaint speech data input into the initial emotion recognition model.

[0288] The multimodal fusion layer is used to: non-linearly integrate the text enhancement features and the speech enhancement features to obtain fused features;

[0289] The emotion classification layer is used to: perform emotion recognition on the fused features and output the emotion recognition result.

[0290] Optionally, the emotion recognition result includes an emotion recognition score, the sentiment tendency result includes a sentiment tendency score, and the complaint escalation prediction result includes a complaint escalation prediction score.

[0291] The step of determining the complaint escalation score of the target user based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result includes:

[0292] The emotion recognition score, the sentiment tendency score, and the complaint escalation prediction score are normalized to obtain a first normalized value corresponding to the emotion recognition score, a second normalized value corresponding to the sentiment tendency score, and a third normalized value corresponding to the complaint escalation prediction score.

[0293] Using preset weight information, the first normalized value, the second normalized value, and the third normalized value are weighted and summed to obtain the complaint escalation score. The preset weight information is used to indicate: a first weight corresponding to the emotion recognition score, a second weight corresponding to the emotion tendency score, and a third weight corresponding to the complaint escalation prediction score. The sum of the first weight, the second weight, and the third weight is 1.

[0294] The apparatus 700 of this application embodiment can perform all the steps of the above-described model training method and escalation complaint behavior prediction method, and achieve the same beneficial effects. To avoid repetition, it will not be described again.

[0295] This application also provides an electronic device. Since the principle by which the electronic device solves the problem is similar to the model training method or the escalation complaint behavior prediction method in this application, the implementation of this electronic device can refer to the implementation of the aforementioned model training method or escalation complaint behavior prediction method; repeated details will not be elaborated further. Figure 8 As shown, the electronic device according to an embodiment of this application includes: a processor 800, configured to read a program from a memory 820 and execute the following processes:

[0296] Multiple training data sets are acquired, including historical user profile data, historical complaint voice data corresponding to the historical user profile data, emotion tags corresponding to the historical complaint voice data, sentiment tendency tags corresponding to the historical complaint voice data, and complaint escalation tags corresponding to the historical user profile data. The complaint escalation tags are used to indicate whether the user corresponding to the historical user profile data should escalate the complaint when the user sends the voice corresponding to the historical complaint voice data.

[0297] Based on the multiple training data sets, an initial model is trained to obtain a target model. The initial model includes an initial emotion recognition model, an initial sentiment analysis model, an initial complaint classification model, and an initial prediction model. The initial emotion recognition model is used to: receive the historical complaint voice data, perform emotion recognition on the historical complaint voice data, and output an emotion recognition result. The initial sentiment analysis model is used to: receive the historical complaint voice data, perform sentiment tendency analysis on the historical complaint voice data, and output a sentiment tendency result. The initial complaint classification model is used to: receive the historical user profile data, predict whether a user will escalate their complaint based on the historical user profile data, and output a complaint escalation prediction result. The initial prediction model is used to: receive the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determine a complaint escalation score based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation score indicates the probability of the corresponding user escalating their complaint.

[0298] Among them, Figure 8 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 800) and memory (memory 820). The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides the interface. Processor 800 is responsible for managing the bus architecture and general processing, and memory 820 can store data used by processor 800 during operation.

[0299] Optionally, the processor 800 is configured to read the program from the memory 820 and execute the following processes:

[0300] The historical complaint voice data in the plurality of training data groups are preprocessed to obtain a first voice data set. The first voice data set includes all historical complaint voice data after preprocessing in the plurality of training data groups. The voice data preprocessing includes at least one of the following: noise reduction, frame segmentation, windowing, and length alignment.

[0301] Each historical complaint voice data after preprocessing in the first voice data set is converted into text to obtain a first text feature set, which includes all text features after text conversion in the first voice data set.

[0302] The historical user profile data in the plurality of training data groups are preprocessed to obtain a first profile data set. The first profile data set includes all historical user profile data after the profile data preprocessing in the plurality of training data groups. The profile data preprocessing includes at least one of the following: handling missing values ​​of numerical features, handling missing values ​​of categorical features, handling extreme missing values, and normalization.

[0303] Based on the emotion tags in the first speech data set, the first text feature set, and the plurality of training data sets, the initial emotion recognition model is trained to obtain a target emotion recognition model; and based on the sentiment tendency tags in the first text feature set and the plurality of training data sets, the initial sentiment analysis model is trained to obtain a target sentiment analysis model; and based on the complaint escalation tags in the first profile data set and the plurality of training data sets, the initial complaint classification model is trained to obtain a target complaint classification model; wherein, the target model includes the target emotion recognition model, the target sentiment analysis model, and the target complaint classification model.

[0304] Optionally, the processor 800 is configured to read the program from the memory 820 and execute the following processes:

[0305] Text features in the first text feature set are preprocessed to obtain a second text feature data set. The text feature preprocessing includes at least one of the following: text cleaning and semantic encoding. At least one preprocessed historical complaint voice data in the first voice data set is subjected to a first enhancement process to obtain at least one enhanced voice data corresponding one-to-one with the preprocessed historical complaint voice data. The first enhancement process includes at least one of the following: adding background noise and fundamental frequency transformation. The at least one enhanced voice data is added to the first voice data set to obtain a second voice data set. At least one first text feature in the second text feature set is subjected to a second enhancement process to obtain at least one first enhanced text feature corresponding one-to-one with the at least one first text feature. The second enhancement process includes at least one of the following: synonym replacement and random insertion or deletion of lexical units. The at least one first enhanced text feature is added to the second text feature set to obtain a third text feature set. Based on the second voice data set, the third text feature set, and the emotion labels in the plurality of training data sets, the initial emotion recognition model is trained to obtain the target emotion recognition model.

[0306] And / or, perform a first preprocessing on the text features in the first text feature set to obtain a fourth text feature set, wherein the first preprocessing includes at least one of the following: text clarity, dialogue structuring, and format conversion; perform a third enhancement processing on at least one of the first preprocessed first text features in the fourth text feature set to obtain at least one second enhanced text feature corresponding one-to-one with the at least one first preprocessed first text feature, wherein the third enhancement processing includes at least one of the following: synonym sentiment word replacement, back-translation enhancement, and generative enhancement; add the at least one second enhanced text feature to the fourth text feature set to obtain a fifth text feature set; and train the initial sentiment analysis model based on the fifth text feature set and the sentiment tendency labels in the plurality of training data sets to obtain the target sentiment analysis model;

[0307] And / or, at least one first profile data in the first profile data set is used to generate virtual samples to obtain at least one first enhanced profile data, wherein the complaint escalation label corresponding to the first profile data is a first label, which is used to indicate that the user has escalated the complaint; the at least one first enhanced profile data is added to the first profile data set to obtain a second profile data set; at least one second profile data is deleted from the second profile data set to obtain a third profile data set, wherein the complaint escalation label corresponding to the second profile data is a second label, and the first label is used to indicate that the user has not escalated the complaint; based on the third profile data set and the complaint escalation labels in multiple training data sets, the initial complaint classification model is trained to obtain the target complaint classification model.

[0308] Optionally, the initial emotion recognition model includes a cross-attention interaction layer, a multimodal fusion layer, and an emotion classification layer;

[0309] The cross-attention interaction layer is used to: enhance text features using speech features to obtain text-enhanced features, and enhance speech features using the text features to obtain speech-enhanced features, wherein the text features are features obtained by text conversion of complaint speech data input into the initial emotion recognition model, and the speech features are features obtained by acoustic feature extraction of complaint speech data input into the initial emotion recognition model.

[0310] The multimodal fusion layer is used to: non-linearly integrate the text enhancement features and the speech enhancement features to obtain fused features;

[0311] The emotion classification layer is used to: perform emotion recognition on the fused features and output the emotion recognition result.

[0312] Optionally, the processor 800 is configured to read the program from the memory 820 and execute the following processes:

[0313] The target user's complaint voice data and user profile data are input into the target model to obtain the emotion recognition result, sentiment tendency result, complaint escalation prediction result, and complaint escalation score output by the target model. The target model is trained using the model training method described above and includes a target emotion recognition model, a target sentiment analysis model, a target complaint classification model, and a target prediction model. The target emotion recognition model receives the complaint voice data, performs emotion recognition on the complaint voice data, and outputs the emotion recognition result. The target sentiment analysis model receives the complaint voice data, performs sentiment tendency analysis on the complaint voice data, and outputs the sentiment tendency result. The target complaint classification model receives the user profile data, predicts whether the user will escalate their complaint based on the user profile data, and outputs the complaint escalation prediction result. The target prediction model receives the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determines the target user's complaint escalation score based on these results. The complaint escalation prediction score indicates the probability that the target user will escalate their complaint.

[0314] Optionally, the emotion recognition result includes an emotion recognition score, the sentiment tendency result includes a sentiment tendency score, and the complaint escalation prediction result includes a complaint escalation prediction score.

[0315] The step of determining the complaint escalation score of the target user based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result includes:

[0316] The emotion recognition score, the sentiment tendency score, and the complaint escalation prediction score are normalized to obtain a first normalized value corresponding to the emotion recognition score, a second normalized value corresponding to the sentiment tendency score, and a third normalized value corresponding to the complaint escalation prediction score.

[0317] Using preset weight information, the first normalized value, the second normalized value, and the third normalized value are weighted and summed to obtain the complaint escalation score. The preset weight information is used to indicate: a first weight corresponding to the emotion recognition score, a second weight corresponding to the emotion tendency score, and a third weight corresponding to the complaint escalation prediction score. The sum of the first weight, the second weight, and the third weight is 1.

[0318] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described model training method or escalation complaint behavior prediction method embodiments, and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0319] This application also provides a computer program product, including computer instructions. When executed by a processor, these computer instructions implement the various processes of the above-described model training method or upgraded complaint behavior prediction method, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0320] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0321] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0322] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A model training method, characterized in that, The method includes: Multiple training data sets are acquired, including historical user profile data, historical complaint voice data corresponding to the historical user profile data, emotion tags corresponding to the historical complaint voice data, sentiment tendency tags corresponding to the historical complaint voice data, and complaint escalation tags corresponding to the historical user profile data. The complaint escalation tags are used to indicate whether the user corresponding to the historical user profile data should escalate the complaint when the user sends the voice corresponding to the historical complaint voice data. Based on the multiple training data sets, an initial model is trained to obtain a target model. The initial model includes an initial emotion recognition model, an initial sentiment analysis model, an initial complaint classification model, and an initial prediction model. The initial emotion recognition model is used to: receive the historical complaint voice data, perform emotion recognition on the historical complaint voice data, and output an emotion recognition result. The initial sentiment analysis model is used to: receive the historical complaint voice data, perform sentiment tendency analysis on the historical complaint voice data, and output a sentiment tendency result. The initial complaint classification model is used to: receive the historical user profile data, predict whether a user will escalate their complaint based on the historical user profile data, and output a complaint escalation prediction result. The initial prediction model is used to: receive the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determine a complaint escalation score based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation score indicates the probability of the corresponding user escalating their complaint.

2. The method according to claim 1, characterized in that, The step of training the initial model based on the multiple training data sets to obtain the target model includes: The historical complaint voice data in the plurality of training data groups are preprocessed to obtain a first voice data set. The first voice data set includes all historical complaint voice data after preprocessing in the plurality of training data groups. The voice data preprocessing includes at least one of the following: noise reduction, frame segmentation, windowing, and length alignment. Each historical complaint voice data after preprocessing in the first voice data set is converted into text to obtain a first text feature set, which includes all text features after text conversion in the first voice data set. The historical user profile data in the plurality of training data groups are preprocessed to obtain a first profile data set. The first profile data set includes all historical user profile data after the profile data preprocessing in the plurality of training data groups. The profile data preprocessing includes at least one of the following: handling missing values ​​of numerical features, handling missing values ​​of categorical features, handling extreme missing values, and normalization. Based on the emotion tags in the first speech data set, the first text feature set, and the plurality of training data sets, the initial emotion recognition model is trained to obtain a target emotion recognition model; and based on the sentiment tendency tags in the first text feature set and the plurality of training data sets, the initial sentiment analysis model is trained to obtain a target sentiment analysis model; and based on the complaint escalation tags in the first profile data set and the plurality of training data sets, the initial complaint classification model is trained to obtain a target complaint classification model; wherein, the target model includes the target emotion recognition model, the target sentiment analysis model, and the target complaint classification model.

3. The method according to claim 2, characterized in that, The step of training the initial emotion recognition model based on the emotion tags in the first speech data set, the first text feature set, and the plurality of training data sets to obtain the target emotion recognition model includes: performing text feature preprocessing on the text features in the first text feature set to obtain a second text feature data set, wherein the text feature preprocessing includes at least one of the following: text cleaning, semantic encoding; and performing a first enhancement processing on at least one preprocessed historical complaint speech data in the first speech data set to obtain at least one enhanced speech data corresponding one-to-one with the at least one preprocessed historical complaint speech data, wherein the first enhancement processing includes at least one of the following: adding background noise, etc. Frequency transformation; adding the at least one enhanced speech data to the first speech data set to obtain a second speech data set; performing a second enhancement process on at least one first text feature in the second text feature set to obtain at least one first enhanced text feature corresponding one-to-one with the at least one first text feature, wherein the second enhancement process includes at least one of the following: synonym replacement, random insertion or deletion of lexical units; adding the at least one first enhanced text feature to the second text feature set to obtain a third text feature set; training the initial emotion recognition model based on the second speech data set, the third text feature set, and the emotion labels in the plurality of training data sets to obtain the target emotion recognition model; And / or, training the initial sentiment analysis model based on the first text feature set and the sentiment tendency labels in the plurality of training data groups to obtain the target sentiment analysis model includes: performing a first preprocessing on the text features in the first text feature set to obtain a fourth text feature set, wherein the first preprocessing includes at least one of the following: text clarity, dialogue structuring, and format conversion; performing a third enhancement processing on at least one of the first preprocessed first text features in the fourth text feature set to obtain at least one second enhanced text feature corresponding one-to-one with the at least one first preprocessed first text feature, wherein the third enhancement processing includes at least one of the following: synonym sentiment word replacement, back-translation enhancement, and generative enhancement; adding the at least one second enhanced text feature to the fourth text feature set to obtain a fifth text feature set; and training the initial sentiment analysis model based on the fifth text feature set and the sentiment tendency labels in the plurality of training data groups to obtain the target sentiment analysis model. And / or, training the initial complaint classification model based on the complaint escalation labels in the first profile data set and the plurality of training data groups to obtain the target complaint classification model includes: generating virtual samples from at least one first profile data in the first profile data set to obtain at least one first enhanced profile data, wherein the complaint escalation label corresponding to the first profile data is a first label, which is used to indicate that the user has escalated the complaint; adding the at least one first enhanced profile data to the first profile data set to obtain a second profile data set; deleting at least one second profile data from the second profile data set to obtain a third profile data set, wherein the complaint escalation label corresponding to the second profile data is a second label, and the first label is used to indicate that the user has not escalated the complaint; and training the initial complaint classification model based on the third profile data set and the complaint escalation labels in the plurality of training data groups to obtain the target complaint classification model.

4. The method according to any one of claims 1 to 3, characterized in that, The initial emotion recognition model includes a cross-attention interaction layer, a multimodal fusion layer, and an emotion classification layer; The cross-attention interaction layer is used to: enhance text features using speech features to obtain text-enhanced features, and enhance speech features using the text features to obtain speech-enhanced features, wherein the text features are features obtained by text conversion of complaint speech data input into the initial emotion recognition model, and the speech features are features obtained by acoustic feature extraction of complaint speech data input into the initial emotion recognition model. The multimodal fusion layer is used to: non-linearly integrate the text enhancement features and the speech enhancement features to obtain fused features; The emotion classification layer is used to: perform emotion recognition on the fused features and output the emotion recognition result.

5. A method for predicting escalating complaint behavior, characterized in that, The method includes: The target user's complaint voice data and user profile data are input into a target model to obtain the target model's output of emotion recognition results, sentiment tendency results, complaint escalation prediction results, and complaint escalation score. The target model is trained using the model training method described in any one of claims 1 to 4. The target model includes a target emotion recognition model, a target sentiment analysis model, a target complaint classification model, and a target prediction model. The target emotion recognition model receives the complaint voice data, performs emotion recognition on the complaint voice data, and outputs the emotion recognition result. The target sentiment analysis model receives the complaint voice data, performs sentiment tendency analysis on the complaint voice data, and outputs the sentiment tendency result. The target complaint classification model receives the user profile data, predicts whether the user will escalate their complaint based on the user profile data, and outputs the complaint escalation prediction result. The target prediction model receives the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determines the target user's complaint escalation score based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation prediction score indicates the probability that the target user will escalate their complaint.

6. The method according to claim 5, characterized in that, The emotion recognition result includes an emotion recognition score, the sentiment tendency result includes a sentiment tendency score, and the complaint escalation prediction result includes a complaint escalation prediction score. The step of determining the complaint escalation score of the target user based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result includes: The emotion recognition score, the sentiment tendency score, and the complaint escalation prediction score are normalized to obtain a first normalized value corresponding to the emotion recognition score, a second normalized value corresponding to the sentiment tendency score, and a third normalized value corresponding to the complaint escalation prediction score. Using preset weight information, the first normalized value, the second normalized value, and the third normalized value are weighted and summed to obtain the complaint escalation score. The preset weight information is used to indicate: a first weight corresponding to the emotion recognition score, a second weight corresponding to the emotion tendency score, and a third weight corresponding to the complaint escalation prediction score. The sum of the first weight, the second weight, and the third weight is 1.

7. An escalation complaint behavior prediction device, characterized in that, The device includes: The acquisition module is used to acquire multiple training data sets, which include historical user profile data, historical complaint voice data corresponding to the historical user profile data, emotion tags corresponding to the historical complaint voice data, sentiment tendency tags corresponding to the historical complaint voice data, and complaint escalation tags corresponding to the historical user profile data. The complaint escalation tags are used to indicate whether the user corresponding to the historical user profile data should escalate the complaint when the user sends the voice corresponding to the historical complaint voice data. The training module is used to train an initial model based on the multiple training data sets to obtain a target model. The initial model includes an initial emotion recognition model, an initial sentiment analysis model, an initial complaint classification model, and an initial prediction model. Specifically, the initial emotion recognition model receives historical complaint voice data, performs emotion recognition on the historical complaint voice data, and outputs an emotion recognition result. The initial sentiment analysis model receives historical complaint voice data, performs sentiment tendency analysis on the historical complaint voice data, and outputs a sentiment tendency result. The initial complaint classification model receives historical user profile data, predicts whether a user will escalate their complaint based on the historical user profile data, and outputs a complaint escalation prediction result. The initial prediction model receives the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result, and determines a complaint escalation score based on the emotion recognition result, the sentiment tendency result, and the complaint escalation prediction result. The complaint escalation score indicates the probability of the corresponding user escalating their complaint. The device further includes: The prediction module is used to input the complaint voice data of the target user and the user profile data of the target user into the target model, and obtain the emotion recognition result, sentiment tendency result, complaint escalation prediction result, and complaint escalation score output by the target model. The target model is a model trained based on the model training method of any one of claims 1 to 4, and includes a target emotion recognition model, a target sentiment analysis model, a target complaint classification model, and a target prediction model. The target emotion recognition model is used to: receive the complaint voice data, perform emotion recognition on the complaint voice data, and output the emotion recognition result; the target sentiment score... The analysis model is used to: receive the complaint voice data, perform sentiment analysis on the complaint voice data, and output the sentiment result; the target complaint classification is used to: receive the user profile data, predict whether the user will escalate the complaint based on the user profile data, and output the complaint escalation prediction result; the target prediction model is used to: receive the emotion recognition result, the sentiment result, and the complaint escalation prediction result, and determine the complaint escalation score of the target user based on the emotion recognition result, the sentiment result, and the complaint escalation prediction result, and the complaint escalation prediction score is used to indicate the probability of the target user escalating the complaint.

8. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when executed by the processor, the computer program implements the steps of the model training method as described in any one of claims 1 to 4, or when executed by the processor, implements the steps of the escalation complaint behavior prediction method as described in any one of claims 5 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the model training method as described in any one of claims 1 to 4, or, when executed by the processor, implements the steps of the escalation complaint behavior prediction method as described in any one of claims 5 to 6.

10. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the model training method as described in any one of claims 1 to 4, or, when executed by the processor, implement the steps of the escalation complaint behavior prediction method as described in any one of claims 5 to 6.

Images