Customer service dialogue user emotion recognition method and system based on multi-dimensional features

Abstract

The application provides a customer service dialogue user emotion recognition method and system based on multi-dimensional characteristics, three types of data of user text flow, non-text tone signal and interactive behavior time sequence data in the customer service dialogue are collected, a three-dimensional characteristic system of text semantics, tone characteristics and interactive behaviors is constructed; standardized features are respectively extracted by using a BERT model and a full connection network, dynamic weights are generated by combining cross-dimensional attention mechanisms, and emotion scores are calculated by using a product nonlinear fusion formula; a three-level grading response strategy of low risk, medium risk and high risk is executed according to the scores, and a recognition, intervention and feedback closed loop is realized. The application solves the problems of single dimension, simple fusion and no customer service exclusive intervention in the prior art, realizes high accuracy, millisecond-level real-time recognition and closed-loop intervention, and significantly reduces the misjudgment rate and the complaint rate.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and intelligent customer service human-computer interaction technology, and more specifically, to a method and system for recognizing user emotions in customer service dialogues based on multi-dimensional features. Background Technology

[0002] In the current field of intelligent customer service human-computer interaction, user emotion recognition technology mainly relies on single-modal data analysis. Its core implementation method is based on semantic mining of text content using natural language processing, or feature extraction of speech tone based on voiceprint recognition. Currently, the mainstream technical solutions can be mainly divided into the following two categories: 1. Deep Learning-Based Text Sentiment Analysis. This technology is widely used in customer service systems in industries such as e-commerce and finance. Its core implementation involves encoding user-input text using pre-trained language models (such as BERT and RoBERTa). Specific steps include: first, segmenting and vectorizing the dialogue text; then, capturing contextual semantics through the model's attention mechanism; and finally, using a classifier to output sentiment labels (such as positive, neutral, and negative). For example, in a bank's intelligent customer service system, when a user inputs "I have a problem with this bill," the system identifies it as a negative sentiment through semantic analysis, thereby triggering an appropriate response strategy.

[0003] 2. Keyword matching technology based on a rule engine. This technology is often used in early customer service systems or scenarios with high real-time requirements. It mainly consists of three modules: a keyword database, a matching engine, and a threshold determination mechanism. The implementation steps are as follows: The system monitors the user's input stream in real time, matching the input text against a pre-defined negative emotion keyword database (such as "complaint," "bad review," "fraud," etc.). When the number of matches or the keyword frequency weight exceeds a set threshold, the system determines that the user is in a negative emotional state. For example, when a user continuously sends messages containing "I want to complain," the system will immediately trigger the process of transferring the user to a live agent.

[0004] While the above-mentioned technical solutions have achieved the basic functions of emotion recognition to a certain extent, they still have the following technical shortcomings in practical applications: (1) Lag and poor scenario adaptability of text semantic analysis. Existing technologies rely on complete sentences or paragraphs for semantic modeling. When faced with colloquial and fragmented customer service dialogue scenarios, misjudgments often occur due to incomplete information. For example, when a user only enters “???” or “…”, due to the lack of clear semantic vocabulary, deep learning-based models usually cannot identify the implied confusion or impatience. When a user enters ironic words such as “hehe”, the model may misjudge it as a positive emotion due to bias in the training data. This lag and poor scenario adaptability result in insufficient real-time emotion recognition, with an average response delay of ≥2 seconds per round of dialogue, and a misjudgment rate as high as 35% in complex contexts, directly affecting the intelligent experience of the customer service system.

[0005] (2) The lack of interactive behavior features leads to a single dimension of emotion assessment. Existing technologies focus only on the text content itself and do not include the user's interactive behavior during the conversation in the scope of emotion recognition. For example, behaviors such as frequently clicking the "transfer to human" button while waiting for the robot to reply, sending multiple text messages in a short period of time, or not entering anything for a long time but not ending the conversation are all important indicators of their emotional state. However, the current system lacks a mechanism for collecting and analyzing this behavioral data, resulting in the inability to fully capture the user's true emotional intentions. This single-dimensional recognition method means that when the user does not use explicit negative words but shows obvious impatience, the system will still judge it as "neutral" emotion, which leads to incorrect matching of service strategies and a decrease in user satisfaction to below 75%.

[0006] (3) The lack of multimodal data fusion leads to high technical implementation costs and insufficient robustness. In scenarios involving voice customer service, existing technologies typically operate text analysis and voiceprint recognition as independent modules, lacking a mechanism for fusing and analyzing multimodal data such as text, voice, and intonation. For example, when a user's voice has a flat intonation but the text contains negative words, or when the text is neutral but the speaking speed is rapid and the volume is increased, the single-modal recognition module may deviate. Due to the lack of a weighted fusion algorithm for multimodal features, existing systems need to deploy multiple independent modules to run in parallel, resulting in an increase in computing resource consumption of ≥40%. Furthermore, when faced with abnormal situations such as noise interference and network latency, the stability of the system decreases significantly, and the accuracy of emotion recognition can fluctuate within ±15%, making it difficult to meet the requirements of high-reliability scenarios.

[0007] The core reason for the above defects is: 1. Limitations of feature extraction mechanisms: Existing feature engineering designs rely too heavily on the semantic information of text and do not introduce quantitative extraction algorithms for non-textual features such as punctuation, input rhythm, and interactive behavior, resulting in the model's inability to capture "paralinguistic" emotional signals in digital communication.

[0008] 2. Data Dimensional Fragmentation: The technical implementation processes text content separately from user behavior, voice features, and other data, lacking a multi-dimensional data fusion framework and failing to design a mapping logic between interactive behavior features and emotion tags, resulting in insufficient comprehensiveness of emotion assessment.

[0009] 3. Unique model architecture: Existing systems mostly use single-modal deep learning models or rule engines, without introducing weighted fusion algorithms for multimodal features (such as cross-modal feature fusion under attention mechanism), which leads to increased computational resource consumption and insufficient robustness, making it difficult to cope with the needs of emotion recognition in complex scenarios.

[0010] Therefore, there is an urgent need for a solution that can address the problems of low recognition accuracy, poor real-time performance, weak scene adaptability, and rigid response strategies. Summary of the Invention

[0011] In view of the above problems, the purpose of this invention is to provide a method and system for user emotion recognition in customer service dialogue based on multi-dimensional features. By integrating text semantic features, tone features and interactive behavior features in a multi-dimensional dynamic fusion method, the accuracy and real-time performance of user emotion recognition in customer service dialogue can be improved, and hierarchical intelligent response can be achieved, thereby reducing the false judgment rate and improving the user experience.

[0012] In a first aspect, the present invention provides a method for recognizing user emotions in customer service dialogues based on multi-dimensional features, applied to electronic devices, comprising the following steps: S1: Collect multi-source data during customer service dialogue in real time and preprocess the multi-source data; wherein, the multi-source data includes user text stream, non-text tone signals and interaction behavior time sequence data; S2: Extract features from the preprocessed multi-source data to obtain multi-dimensional standardized features, including text semantic features, tone features, and interactive behavior features; S3: Based on a pre-built cross-dimensional attention dynamic fusion model, a weighted fusion of the multi-dimensional standardized features is performed using a product-based nonlinear fusion algorithm to generate a comprehensive sentiment score; wherein, the product-based nonlinear fusion formula is: F = min (S) text ×(1+α S tone )×(1+β S βehavior ),100) Among them, S text For text semantic features, S tone As a feature of tone, S βehavior For interactive behavior features, α and β are the adjustment coefficients of the cross-dimensional attention dynamic fusion model, and their values ​​are both in the range of [0,2]. S4: Determine the emotional level of the comprehensive emotional score according to the preset emotional level classification standard, and execute the graded response strategy corresponding to the emotional level; wherein, the preset emotional level classification standard divides the user's comprehensive emotions into three levels: low risk, medium risk, and high risk.

[0013] In addition, an optional technical solution is that the non-text tone signal includes continuous punctuation, repeated characters, input rhythm, and character capitalization; The interactive behavior time-series data is customer service scenario-specific behavior data, including interruption behavior when the robot does not reply, clicking to transfer to human agent behavior, short-term continuous communication behavior, long-term silence behavior, and like or dislike feedback behavior.

[0014] In addition, an optional technical solution is to preprocess the multi-source data, including: Cleaning process is used to remove noise and invalid content from the multi-source data; Normalization is used to unify the format, precision, and value range of the multi-source data; Quantization processing is used to transform non-textual data and behavioral data in the multi-source data into machine-recognizable structured features.

[0015] In addition, an optional technical solution is that the text semantic features are extracted and normalized to the [0,100] interval through a BERT or RoBERTa pre-trained language model; the tone features are extracted and normalized to the [0,1] interval through a preset fully connected network; and the interaction behavior features are extracted and normalized to the [0,1] interval through a preset weighted model.

[0016] In addition, an optional technical solution is that the cross-dimensional attention dynamic fusion model maps the multi-dimensional standardized features to a unified dimensional space, calculates the feature correlation degree through a preset Transformer encoder, and dynamically generates the adjustment coefficients α and β.

[0017] Alternatively, the preset emotion level classification criteria and corresponding response strategies are as follows: Low-risk range: F < 30, implement standard customer service response strategy; Medium risk range: 30≤F<60, automatically insert empathic remarks and implement reassurance response strategies; High-risk zone: F≥60, force transfer to human agent and push emotional warning, implement upgraded intervention strategy.

[0018] Alternatively, when any dimension of the multi-dimensional standardized features is missing or subject to noise interference, the attention weight of the missing dimension is automatically reduced, and compensation calculation is performed based on the remaining dimension features.

[0019] Secondly, the present invention also provides a customer service dialogue user emotion recognition system based on multi-dimensional features, comprising: The data acquisition unit is used to collect multi-source data in real time during customer service dialogues and to preprocess the multi-source data; wherein, the multi-source data includes user text streams, non-text tone signals, and interactive behavior time-series data; The feature extraction unit is used to extract features from the preprocessed multi-source data to obtain multi-dimensional standardized features, including text semantic features, tone features, and interactive behavior features. The weighted fusion unit is used to perform weighted fusion of the multi-dimensional standardized features based on a pre-built cross-dimensional attention dynamic fusion model, using a product-based nonlinear fusion algorithm to generate a comprehensive sentiment score; wherein the product-based nonlinear fusion formula is: F = min (S) text ×(1+α S tone )×(1+β S βehavior ),100) Among them, S text For text semantic features, S tone As a feature of tone, S βehavior For interactive behavior features, α and β are the adjustment coefficients of the cross-dimensional attention dynamic fusion model, and their values ​​are both in the range of [0,2]. The rating unit is used to determine the emotional level of the comprehensive emotional score according to a preset emotional rating standard, and to execute a graded response strategy corresponding to the emotional level; wherein the preset emotional rating standard divides the user's comprehensive emotions into three levels: low risk, medium risk, and high risk.

[0020] Thirdly, the present invention provides an electronic device, the electronic device including a memory, a processor, and a customer service dialogue user emotion recognition program based on multi-dimensional features stored in the memory and executable on the processor, wherein the customer service dialogue user emotion recognition program based on multi-dimensional features, when executed by the processor, implements the customer service dialogue user emotion recognition method based on multi-dimensional features as described above.

[0021] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the customer service dialogue user emotion recognition method based on multi-dimensional features as described above.

[0022] This invention discloses a method, system, and storage medium for customer service dialogue user emotion recognition based on multi-dimensional features. By constructing a three-dimensional feature system of user text stream, non-text tone signals, and interactive behavior time-series data, and employing cross-dimensional attention dynamic fusion and product-type nonlinear algorithms to accurately capture nonlinear changes in emotions, it effectively solves the shortcomings of existing technologies such as single dimension, simple fusion, and insufficient scenario adaptability. It significantly improves the accuracy of emotion recognition in customer service scenarios and reduces the false judgment rate. At the same time, it achieves millisecond-level real-time response and hierarchical closed-loop intervention, effectively intercepting potential complaints and improving user satisfaction and service intelligence.

[0023] To achieve the foregoing and related objectives, one or more aspects of the invention include the features that will be described in detail below. The following description and accompanying drawings illustrate certain exemplary aspects of the invention. However, these aspects indicate only a few of the various ways in which the principles of the invention can be used. Furthermore, the invention is intended to encompass all such aspects and their equivalents. Attached Figure Description

[0024] Other objects and results of the invention will become more apparent and readily understood with reference to the following description taken in conjunction with the accompanying drawings. In the drawings: Figure 1 This is a flowchart illustrating a customer service dialogue user emotion recognition method based on multi-dimensional features according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall technical architecture of the customer service dialogue user emotion recognition method based on multi-dimensional features provided in an embodiment of the present invention; Figure 3 A schematic diagram of a customer service dialogue user emotion recognition system based on multi-dimensional features according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the internal structure of an electronic device provided according to an embodiment of the present invention.

[0025] In all the accompanying drawings, the same reference numerals indicate similar or corresponding features or functions. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0028] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0029] To better illustrate the technical solution of the present invention, some of the technical terms involved in the present invention will be briefly explained below.

[0030] Multi-dimensional features: refers to the simultaneous collection of three types of emotion-related information during customer service conversations: text semantics, tone features, and interactive behavior. This is used to comprehensively represent the user's emotional state, which is different from information collection methods based on a single text dimension.

[0031] Text semantics refers to the semantic content and emotional tendency carried by the text entered by the user. Semantic parsing and sentiment classification are performed by natural language processing models (such as BERT and RoBERTa), which is the basic dimension for identifying user emotions.

[0032] Non-textual tone signals: These refer to non-textual paralinguistic signals in user input, including continuous punctuation, repeated characters, input rhythm, character capitalization, etc., which are used to capture implicit emotions in colloquial and fragmented expressions.

[0033] Interactive behavior: refers to the time sequence data of user actions during the conversation, including frequent clicks to switch to human assistant, interrupting replies, delayed responses, repeated sending, topic dominance, etc., which is an important behavioral dimension reflecting the intensity of user emotions.

[0034] Feature quantization refers to the process of converting unstructured information such as text, tone, and interactive behavior into standardized numerical values ​​(such as 0 to 1 or 0 to 100), so that multi-source data can participate in unified calculation and fusion analysis.

[0035] Attention mechanism: refers to the technique of dynamically assigning weights to each feature based on the context, which can automatically identify the importance of various emotional features in different scenarios and improve the recognition adaptability.

[0036] Dynamic fusion algorithm: refers to a product-based fusion method based on attention mechanism. It achieves the coordinated fusion of text, tone and interactive behavior features by dynamically adjusting coefficients, amplifying key emotional signals, which is different from the static fusion method with fixed weights.

[0037] Emotional score: refers to a score of 0 to 100 output after the fusion of multi-dimensional features. The higher the score, the stronger the user's negative emotions. It is used to classify and determine the level of emotion.

[0038] Tiered response strategy: This refers to triggering differentiated service strategies based on emotion score ranges, including normal response for low-risk cases, reassurance messages for medium-risk cases, mandatory transfer to human agent and supervisor warning for high-risk cases, to achieve a closed-loop linkage between emotion recognition and service intervention.

[0039] Recall rate: refers to the proportion of genuine negative emotion samples that are correctly identified, used to measure the comprehensiveness of emotion recognition; False positive rate: refers to the proportion of emotion recognition results that do not match reality, used to measure the accuracy of recognition.

[0040] Paralinguistics: refers to non-verbal signals that convey emotions beyond the text content, including punctuation, rhythm, and behavioral actions, and is a key dimension of emotion recognition supplemented by this invention.

[0041] This invention focuses on emotion recognition and hierarchical response technology that integrates multi-source data fusion of user text streams, non-text tone signals, and interactive behavior time-series data. It provides a user emotion recognition method and system that integrates multi-dimensional features in customer service dialogue scenarios.

[0042] To describe in detail the user emotion recognition method, system, and storage medium for customer service dialogue based on multi-dimensional features of the present invention, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0043] Example 1

[0044] Figure 1 and Figure 2 The flowchart and overall technical architecture of the customer service dialogue user emotion recognition method based on multi-dimensional features according to embodiments of the present invention are shown respectively. Figure 1 and Figure 2 As shown in the figure, the customer service dialogue user emotion recognition method based on multi-dimensional features provided in this embodiment mainly includes S1-S4.

[0045] S1: Collect multi-source data during customer service conversations in real time and preprocess the multi-source data.

[0046] This step is primarily implemented based on the data acquisition module. The multi-source data collected includes user text streams (also known as text semantics, hereinafter the same) in customer service dialogue scenarios, non-text tone signals (also known as tone symbols, hereinafter the same), and interactive behavior time-series data. Specifically, non-text tone signals can include data such as continuous punctuation, repeated characters, input rhythm, and character capitalization. Interactive behavior time-series data can be customer service scenario-specific behavioral data, including interruption behavior when the robot does not respond, clicking to transfer to human agent behavior, short-term continuous typing behavior, long-term silence behavior, and like or dislike feedback behavior.

[0047] It should be noted that in practical applications, the specific content of multi-source data is not limited to the data mentioned above, and can also be set according to specific application needs.

[0048] In one specific embodiment of the present invention, the data acquisition module captures real-time end-to-end user interaction data through front-end embedding and back-end monitoring. Specifically, this includes: acquiring the text stream input by the user; extracting non-text symbols (such as consecutive punctuation marks and repeated characters); and monitoring interactive events (such as clicking "transfer to human agent," interruptions during the robot's response by sending new messages, and likes / dislikes). Subsequently, the module uses regular expression cleaning and word segmentation tools to transform the unstructured data into standardized feature vectors, providing input for subsequent calculations.

[0049] In terms of technical architecture, the front-end event tracking layer uses a lightweight event tracking SDK (implemented based on JavaScript), which is integrated into the customer service front-end page. A specific implementation example is as follows: 1. Text Data Acquisition Interface: This interface listens for the input event of the user input box, employing a debouncing strategy to capture the current text content every 500ms. An example code snippet is shown below: `document.getElementById('chat-input').addEventListener('input', debounce(captureText, 500))` The data format example is as follows: {user_id: String, text: String, timestamp: Long, session_id: String} 2. The interaction behavior collection interface is implemented through click behavior listening and key element ID mapping. Specifically, it listens for the click events of all button elements using an event bubbling mechanism, mapping `btn_transfer_manual` to "transfer to manual" button, `btn_like` to "like" button, and `btn_dislike` to "dislike" button. An example of the data format is as follows: {user_id: String, action_type: String, element_id: String, timestamp:Long} 3. Timestamp synchronization mechanism: The front-end uses local timestamps plus server time calibration, with a calibration frequency of every 5 minutes; the time precision is limited to milliseconds for calculating input rhythm characteristics. The back-end listening layer employs a distributed architecture to handle high-concurrency data streams, as illustrated in the following example: 3.1 In the message queue layer, Apache Kafka is used as the message middleware, and the data is partitioned by session_id hash. Real-time data is retained for 7 days for feature calculation, and historical data is persisted to MySQL.

[0050] 3.2 The real-time computing layer adopts the Flink stream processing framework, with a sliding window strategy (window size of 5 minutes, sliding step size of 1 minute). Feature computation latency: P99 latency <200ms

[0051] 3.3 At the data storage layer, real-time feature vectors are stored through a Redis cluster with the key name format: feature:{session_id}; historical conversation data is persisted through a MySQL cluster, with table structures containing fields such as user_id, session_id, text, behavior_events, and emotion_score.

[0052] In this embodiment, a WebSocket persistent connection is used to achieve real-time bidirectional communication between the front-end and the back-end. The unified message format is JSON, encoded in UTF-8. A heartbeat packet is sent every 30 seconds, and automatic reconnection occurs after a 60-second timeout.

[0053] The data acquisition module in this embodiment is specifically designed for customer service dialogue scenarios. The specific acquisition and preprocessing rules are as follows: Text data acquisition: Obtain complete text input from users, filter out meaningless characters, and retain keywords and interrogative sentences; Extraction of tone marks: consecutive question marks (???), consecutive exclamation marks (!!!), ellipses (……), and ironic words; regular expression matching rule: ([?]{2,}|[!]{2,}|[.]{3,}); Interaction behavior collection (unique to customer service scenarios): Interruption behavior: If the user sends a new message before the bot responds, it is marked as 1; otherwise, it is marked as 0. Transfer to manual click: If a single conversation receives ≥1 click, it is marked as 1; otherwise, it is marked as 0. Consecutive sending behavior: If ≥2 messages are sent within 3 seconds, it is marked as 1; otherwise, it is marked as 0. Silent behavior: If there is no input for more than 15 seconds and the conversation has not ended, it is marked as 1; otherwise, it is marked as 0. Feedback behavior: Like / Dislike, Dislike is marked as 1; otherwise, it is marked as 0. Preprocessing quantization: Unstructured behavioral data is normalized to values ​​in the range [0,1], and missing data is automatically marked and a feature compensation mechanism is initiated.

[0054] Because the original multi-source data collected is mixed in data type, inconsistent in format, subject to severe noise interference, uneven data quality, large differences in information dimensions, missing and outlier values, and highly unstructured, directly inputting it into the model will lead to semantic ambiguity, feature failure, a significant decrease in recognition accuracy, low computational efficiency, poor model stability, and inability to achieve cross-dimensional fusion. Therefore, it is necessary to clean, standardize, and structure the data through preprocessing in order to support subsequent 3D feature extraction, dynamic fusion, and accurate emotion recognition.

[0055] In one specific embodiment of this invention, the preprocessing of multi-source data mainly includes the following three types: (1) Cleaning process, used to remove noise and invalid content from multi-source data. This includes, but is not limited to: using regular expressions to remove HTML tags, special symbols, garbled characters, and invalid spaces from the text stream, while retaining Chinese characters, English characters, numbers, and valid punctuation; filtering invalid characters (such as pure emoticons and garbled strings) and noise fragments from non-textual tone signals; and removing duplicate events, invalid clicks, and abnormal timestamps from the interactive behavior time-series data.

[0056] (2) Normalization processing is used to unify the format, precision, and value range of multi-source data, unify the format (JSON) and align the timestamps of scattered text, tone, and behavior data, so as to facilitate subsequent three-dimensional feature extraction and fusion. This includes, but is not limited to: uniformly encoding the text stream into UTF-8 format; aligning tone signals by time window and normalizing the amplitude to the [0,1] interval; and unifying the timestamp precision and aligning the session cycle of interactive behavior time series data.

[0057] (3) Quantization processing is used to convert non-textual data and behavioral data in multi-source data into machine-recognizable structured features. This includes, but is not limited to: quantizing non-textual tone signals into three types of structured identifiers: continuous punctuation, repeated characters, and ironic words; and quantizing interactive behavior time-series data into binary features (0 / 1) to form a standardized behavioral feature sequence.

[0058] In addition, it can automatically mark and compensate for missing data to prevent recognition failure due to incomplete data and improve stability under high concurrency.

[0059] The above preprocessing transforms the messy, noisy, and inconsistent raw data into standardized, clean data that can be directly used for feature extraction and model input, thus ensuring the accuracy, speed, and stability of subsequent emotion recognition.

[0060] S2: Feature extraction is performed on the preprocessed multi-source data to obtain multi-dimensional standardized features, including text semantic features, tone features, and interactive behavior features.

[0061] This step can be implemented based on the feature extraction module. In this embodiment, the multi-dimensional standardized features are standardized features in three dimensions: text semantics, tone, and interaction behavior, corresponding to the three extracted source data dimensions: user text stream, non-text tone signals, and temporal data of interaction behavior. As an example, text semantic features can be extracted using a BERT or RoBERTa pre-trained language model and normalized to the [0,100] interval; tone features can be extracted using a pre-set fully connected network and normalized to the [0,1] interval; and interaction behavior features can be extracted using a pre-set weighted model and normalized to the [0,1] interval.

[0062] S3: Based on a pre-built cross-dimensional attention dynamic fusion model, the multi-dimensional standardized features are weighted and fused using a product-type nonlinear fusion algorithm to generate a comprehensive emotion score.

[0063] This step is implemented based on a weighted fusion module. Unlike traditional linear weighting, this invention uses a product-based nonlinear fusion algorithm to calculate the user's comprehensive sentiment score, effectively capturing the synergistic effect between features. In customer service scenarios, text, tone, and behavior are not simply parallel contributors, but rather exhibit a synergistic effect of triggering and amplifying. For example, when the text semantics are neutral (S... text (lower), but the interaction behavior is characterized by high-frequency interruption (S) behavior When the score is extremely high, if linear weighting is used, neutral text will dilute the negative score of the behavior; while if product-based interaction is used, the behavioral features act as an amplification factor (1+β·S). behavior It can accurately capture the abrupt transformation of neutral questions into negative emotions due to impatience while waiting, a non-linear emotional transition feature that linear weighting cannot achieve.

[0064] The product-type nonlinear fusion formula used in this invention is as follows: F = min (S) text ×(1+α S tone )×(1+β S behavior ),100) Among them, S textThe text semantic features are output by the BERT-base-Chinese model and normalized to [0,100]. The higher the value, the stronger the negative sentiment. S tone The tone features are output by the tone feature encoder and normalized to [0,1]. They include continuous punctuation, input rhythm, and ironic words. S behavior The interactive behavior features are output by the interactive behavior encoder and normalized to [0,1], including interruption, transfer to human assistant, continuous firing, silence, and downvoting. α and β are the adjustment coefficients of the cross-dimensional attention dynamic fusion model. They are not fixed values ​​and both range from [0,2]. In the customer service scenario, α=1.2 and β=1.5 by default.

[0065] The cross-dimensional attention dynamic fusion model maps the multi-dimensional standardized features to a unified dimensional space, calculates the feature correlation degree through a preset Transformer encoder, and dynamically generates the adjustment coefficients α and β.

[0066] Specifically, as an example, the cross-dimensional attention dynamic fusion model maps text semantic features, tone features, and interaction behavior features to the same high-dimensional unified feature space through independent linear projection layers; the mapped three types of features are concatenated to form a joint feature sequence; the multi-head self-attention mechanism of the Transformer encoder is used to calculate the association weight matrix between the three types of features and output the feature interaction information; based on the feature association weight matrix, the fully connected layer is used to dynamically generate adjustment coefficients α and β with values ​​ranging from [0,2], where α represents the text-tone association strength and β represents the text-interaction behavior association strength.

[0067] More specifically, the steps for generating dynamic adjustment coefficients α and β through a cross-dimensional attention mechanism are illustrated below: S31: Feature Embedding. This involves embedding the textual semantic features S... text Sentence characteristics tone and interactive behavior characteristics S behavior Mapped to a 128-dimensional unified embedding space; S32: Attention Scoring. A preset scoring function is used to calculate the correlation weights between three features (text, tone, and behavior), dynamically generating α and β coefficients. This allows the cross-dimensional attention dynamic fusion model to focus on the features most relevant to emotion. In a specific embodiment of this invention, the scoring function used is as follows: ; Where Q represents the query vector, which is the query representation of the current feature. In this embodiment, it is specifically a 128-dimensional query vector obtained by linear transformation of text, tone, and behavior features, used to actively match and pay attention to information of other features; K represents the key vector, which is the representation of the feature to be matched with the same dimension as Q. In this embodiment, it is specifically a vector obtained by linear transformation of text, tone, and behavior features, used to be queried by Q to calculate similarity; V represents the 128-dimensional feature value vector corresponding to K, which is the actual information vector corresponding to the feature. In this embodiment, it is specifically the original embedding vector of text, tone, and behavior features, used to be weighted by attention weights to output the fused features. Softmax(·) is the normalization exponential function. This represents the dot product similarity matrix between the query vector and the key vector, used to calculate the correlation between any two features. The larger the value, the stronger the correlation. The square root of the dimension of the key vector ( (where K is the dimension), which serves as a scaling factor to prevent scaling in high dimensions. Excessive numerical values ​​can lead to Softmax saturation and gradient vanishing. V represents the weighted sum of the weight matrix and the value vector, used to filter and enhance key features with attention weights, and output the fused feature vector, which is used to dynamically generate adjustment coefficients α and β.

[0068] S33: Weighted Output. The attention score is mapped to α and β. A higher text-tone correlation increases α, and a higher text-behavior correlation increases β. S34: Scenario Adaptation. For e-commerce, finance, and energy industries, fixed coefficients can be used, such as α=0.8 and β=1.8 for finance scenarios and α=1.2 and β=1.5 for e-commerce scenarios.

[0069] In this embodiment, a phased joint training strategy is adopted to train the cross-dimensional attention dynamic fusion model. The text semantic encoder, tone feature encoder, and interaction behavior encoder are pre-trained independently, and then the cross-dimensional attention fusion network is jointly fine-tuned to ensure that the three-dimensional feature expression capability and cross-dimensional fusion effect are optimal.

[0070] Specifically, as an example, during the text semantic encoder training phase, based on the BERT-base-Chinese pre-trained model, a dataset of 100,000 historical customer service dialogues labeled with negative sentiment probabilities was used for semantic model fine-tuning. Cross-entropy was used as the loss function, the learning rate was set to 2e-5, the batch size to 32, and the training epochs to 3, so that the model could be adapted to colloquial and fragmented customer service texts, and the output negative sentiment score Stext was normalized to the range of 0–100.

[0071] In the training phase of the tone feature encoder, a three-layer fully connected network structure is adopted, with input consisting of three types of quantized features: punctuation, input rhythm, and irony. The training objective is to label tone and emotion. A ReLU activation function and a sigmoid output are used. Batch training is employed, and the mean squared error loss is minimized. The output is a tone feature score S normalized to the 0–1 interval. tone .

[0072] During the training phase of the interactive behavior encoder, a three-layer fully connected network structure is adopted. The input consists of five binary behavioral features: interruption, switching to human intervention, continuous firing, silence, and feedback. The training objective is to label the behavioral emotions. The ReLU activation function and Sigmoid output are used. The weights are optimized through supervised learning, and the output is an interactive behavior feature score S normalized to the 0–1 interval. behavior .

[0073] During the training phase of the cross-dimensional attention fusion network, S text S tone S behavior Mapped to a 128-dimensional unified embedding space, a Transformer encoder structure (including multi-head self-attention, 4-head attention mechanism, and 256-dimensional hidden layer) is constructed. Using the true label of emotion score as the supervision signal, the attention weights are optimized through backpropagation, and the dynamic adjustment coefficients α and β (with values ​​of 0–2) are output. The training process adopts a joint fine-tuning method to achieve adaptive weighted fusion of three-dimensional features.

[0074] After the cross-dimensional attention dynamic fusion model is deployed, incremental training is triggered every 10,000 new dialogue data points, dynamically updating the parameters of the feature extraction network and the fusion network. At the same time, the emotion rating threshold is re-optimized quarterly based on business data to ensure that the model continues to adapt to business changes and maintains recognition stability and accuracy.

[0075] The following will illustrate the formula for calculating sub-features with more specific examples.

[0076] I. Calculation of Sub-dimensions of Tone Features

[0077] 1. Punctuation score calculation: Formula: Punctuation score = (Number of consecutive question marks × 0.4 + Number of consecutive exclamation marks × 0.5 + Number of ellipses × 0.3) / 5; Value range: [0, 1] Calculation example: User inputs "??? I want a refund!!!": Punctuation score = (3 × 0.4 + 3 × 0.5) / 5 = 0.54 User inputs "Hello": Punctuation score = 0 User inputs "Why hasn't it arrived yet...": Punctuation score = (0×0.4 + 0×0.5 + 6×0.3) / 5 = 0.36 2. Input rhythm score calculation: Formula: Input rhythm score = 1 exp(-Δt_avg / 3) Where Δt_avg is the average interval (in seconds) between adjacent messages.

[0078] Value range: [0, 1] Calculation example: The user sends 3 messages within 2 seconds, with an average interval of 1 second: Score = 1 exp(-1 / 3) = 0.28 User sends messages at 10-second intervals: Score = 1 exp(-10 / 3) = 0.96 3. Calculation of Irony Score: Based on matching with an ironic vocabulary database, which includes: "hehe", "haha" (not an emoji), "interesting", etc. Formula: Irony Score = Number of Matched Ironic Words × 0.5 Value range: [0, 1] Calculation example: User inputs "Hehe, that's interesting": Irony score = 2 × 0.5 = 1.0 (maximum truncation) User inputs "OK": Irony score = 0 II. Calculation of Interaction Behavior Sub-dimensions: Each behavior score is binarized (0 or 1), and S is obtained by weighted summation. behavior : Interruption behavior score: 1 point is awarded if the user sends a new message before the robot has finished replying; Points awarded for transferring to human agent: 1 point is awarded for clicking the "transfer to human agent" button ≥ 1 time in a single conversation; Consecutive Message Score: 1 point is awarded for sending ≥2 messages within 3 seconds; Silence score: 1 point is awarded for more than 15 seconds of no input and without ending the conversation; Feedback score: 1 point is awarded for each user who clicks the "dislike" button; Calculation Example: During a conversation: 1 interruption + 2 transfers to human operator + consecutive messages + 10 seconds of silence + 1 dislike (S) behavior = 0.3×1 + 0.2×1 + 0.2×1 + 0.2×0 + 0.1×1 = 0.8.

[0079] III. Explanation of the source of weighting coefficients: Tone feature weights (0.4, 0.3, 0.3): Based on feature importance analysis of 1000 labeled data points, punctuation features contribute the most to the prediction of negative sentiment; Interaction behavior weights (0.3, 0.2, 0.2, 0.2, 0.1): Based on the feature importance ranking of the random forest model, the interruption behavior has the highest weight.

[0080] S4: Determine the emotional level of the comprehensive emotional score according to the preset emotional level classification standard, and execute the graded response strategy corresponding to the emotional level.

[0081] This step is implemented based on a rating module, which uses an emotion score F to implement a three-tiered customer service-specific intervention strategy. The threshold determination criteria and response examples for the preset emotion rating standards are as follows: First, the threshold was determined based on historical data analysis. The data source for determining the threshold was 100,000 historical customer service dialogue records, manually labeled with emotional levels (stable / annoyed / intensely dissatisfied). The optimization objective was to maximize the F1 score (the harmonic mean of recall and precision). Using a grid search with a step size of 5, the following optimization results were obtained: Low-risk threshold 30: 92% of conversations below this value are marked as "stable"; High-risk threshold 60: In conversations exceeding this value, 89% are marked as "strongly dissatisfied"; Medium risk zone [30, 60): The emotions in conversations in this zone fluctuate greatly, requiring proactive intervention.

[0082] Therefore, this embodiment sets the following emotion level classification criteria, i.e., the corresponding graded response rules: 1. Low-risk interval (F < 30): Judgment criteria: The user's emotions are stable, and no negative characteristics are triggered; Response strategy: Implement standard customer service responses and maintain normal service procedures; Example scenario: User typed: "When will my package arrive?" Feature analysis: S text =15 (neutral text), S tone =0 (no tone mark), S behavior =0 (No abnormal behavior) Emotional score: F = 15 × 1 × 1 = 15 System response: "Hello, your package is expected to arrive tomorrow, order number XXX. Please feel free to contact us if you have any questions." 2. Medium-risk interval (30≤F<60): Judgment criteria: The user was irritable and exhibited slightly negative characteristics; Response strategy: Automatically insert empathic reassurance phrases and increase response priority (move the queue forward by 2 places); Example scenario: The user typed: "Why hasn't it arrived yet??? I've been waiting for a week," and then sent three messages within two seconds. Feature analysis: S text =45 (negative text), S tone =0.54 (continuous punctuation), S behavior =0.6 (continuous issuance behavior) Emotional score: F = 45×(1+1.2×0.54)×(1+1.5×0.6) = 45×1.65×1.9 = 141 → Truncated to 100 → Actual treatment as high risk; Corrected example: User input: "Why hasn't it arrived yet???", send a second feature analysis after 5 seconds: S text =40, S tone =0.36, S behavior =0.28 (Input rhythm score); Emotional score: F = 40×(1+1.2×0.36)×(1+1.5×0.28) = 40×1.43×1.42 = 81 → Still high risk; Mid-risk real-world example: User inputs: "The delivery is too slow," clicks to transfer to human operator. Feature analysis: S text =35, S tone =0, S behavior =0.2 (score converted to manual scoring) Emotional score: F = 35 × 1 × (1 + 1.5 × 0.2) = 35 × 1.3 = 45.5 System response: "I'm very sorry to have kept you waiting. I understand your anxiety. I have prioritized your service and am currently checking the logistics information for you..." 3. High-risk zone (F≥60): Judgment criteria: Strong user dissatisfaction, with multiple negative characteristics overlapping; Response strategy: Immediately trigger a forced transfer to a human agent, skipping the chatbot conversation; send an emotion warning notification to the customer service supervisor. The warning includes: user ID, emotion score, core question, and historical interaction behavior.

[0083] Example scenario: The user typed: "This service is terrible!!! I want to complain!!!", clicked to be transferred to a human operator 3 times, and then left a negative comment. Feature analysis: S text =75 (Strongly negative text), S tone =0.6 (consecutive exclamation marks), S behavior=0.9 (transferred to manual review + downvote); Emotional Score: F = 75×(1+1.2×0.6)×(1+1.5×0.9) = 75×1.72×2.35 = 303 → Truncated to 100. System response: Immediately transfer to a human agent and send an alert to the customer service supervisor: "[Emotional Alert] User ID: U123456, Emotional Score: 100 (High Risk), Core Issue: Service Complaint, Historical Interactions: Transferred to human agent 3 times, downvoted feedback, recommended for priority handling."

[0084] In addition, the response results (manual processing results, user satisfaction feedback) can be further synchronized to the data acquisition module, and the cross-dimensional attention dynamic fusion model can be retrained every 10,000 new data points so that the cross-dimensional attention dynamic fusion model can be iteratively optimized regularly; the threshold can also be set to be re-optimized every quarter based on business data so that the threshold can be dynamically adjusted regularly.

[0085] As can be seen from the above embodiments, the customer service dialogue user emotion recognition method based on multi-dimensional features provided by the present invention constructs a three-dimensional feature system of user text stream, non-text tone signals, and interactive behavior time-series data specific to customer service scenarios, and collects customer service-specific features such as interruption, transfer to human agent, continuous messaging, silence, and tone symbols; it pioneers a product-based cross-dimensional dynamic fusion formula to replace the traditional linear weighting, capturing the synergistic effect of features; and it adapts to different industry scenarios based on dynamic weight adjustment of the attention mechanism, without the defects of static weights; thus forming a closed loop of emotion recognition, hierarchical early warning, and proactive intervention. A three-level response strategy is designed for customer service scenarios to achieve proactive complaint interception. When any dimension of data is missing, the weight of that dimension is automatically reduced to ensure the stability of recognition.

[0086] Example 2

[0087] In this embodiment, the hardware implementation environment for applying the present invention includes a server (CPU: Intel Xeon, memory: 16GB) and a customer service terminal; the software environment includes Python 3.8, TensorFlow 2.4 (for semantic analysis), and a MySQL database; the data environment consists of 100,000 historical dialogue data from a certain e-commerce customer service system, containing a library of 200+ negative keywords.

[0088] The specific implementation steps are as follows: 2.1 Data Collection and Preprocessing: User A inputs "What's wrong with this product??? I want a refund!!!", sends two messages within 2 seconds, clicks "transfer to human agent" once, and there is no silence or downvote. 2.2 Three-dimensional feature extraction: Text semantic score Stext: BERT model output 45; Tone feature Stone: continuous punctuation score 0.7 + input rhythm 0.6, weighted Stone=0.65; Interaction behavior score Sbehavior: interruption 0 + transfer to human 1.0 + continuous messaging 1.0 + silence 0 + feedback 0, weighted Sbehavior=0.8; 2.3 Determination of dynamic coefficients: For e-commerce scenarios, α=1.2 and β=1.5; 2.4 Calculation of Emotional Score: F=min(45×(1+1.2×0.65)×(1+1.5×0.8),100)=min(45×1.78×2.2,100)=min(176.22,100)=100; 2.5 Tiered Response: F=100≥60, falling into the high-risk zone, immediately triggering forced transfer to human agent + emotion warning push.

[0089] After implementation of this embodiment, a large number of users who expressed dissatisfaction due to punctuation marks were successfully identified. Compared with the old system, the interception rate of high-risk emotions increased by 50%, and the timely intervention rate of customer service supervisors increased from 60% to 95%.

[0090] Example 3

[0091] The main difference between Example 3 and Example 2 lies in the different scenarios and feature focuses. Example 3 is applied to a financial customer service scenario, where users typically express themselves rationally and use fewer punctuation marks. The dynamic coefficients α and β in the product-based fusion formula are automatically adjusted according to the context of the financial scenario: compared to the e-commerce scenario, the system relatively improves the text semantic features S. text and interactive behavior characteristics S behavior The contribution of reducing the tone feature S tone The weights are adjusted accordingly. For example, after fine-tuning using financial scenario data, the dynamic coefficients are adjusted to α=0.8 (the amplification effect of tone is weakened) and β=1.8 (the amplification effect of behavior is strengthened).

[0092] In addition, this embodiment expands the tone features by adding recognition rules for "all-caps English" (such as "IAMANGRY"). Such strings are marked as special tone events during the preprocessing stage and encoded into Stone (with a value increase of 0.3); the definition of interactive behavior is also expanded by adding a "dialogue round" feature. If the user fails to resolve the issue within 5 rounds of dialogue and the emotion score continues to rise, then in Stone... behavior Add an increment of 0.2 to the system, and the system will automatically upgrade the service priority (e.g., switch to manual intervention in advance).

[0093] Through experimental verification, this invention transforms previously overlooked punctuation marks and interactive behaviors into effective emotional feature data, increasing data utilization by 60%. By fusing three-dimensional features, compared to single text recognition technology, the accuracy of negative emotion recognition increases from 70% to over 95%, while the false positive rate decreases to below 5%. Furthermore, it enables millisecond-level real-time monitoring, with an average response time of ≤1 second from the generation of negative emotions by the user to the triggering of intervention strategies, improving efficiency by 90% compared to manual detection. Through early warning and tiered response, it effectively intercepts potential complaints, with actual application data showing a 40% reduction in customer complaint rates.

[0094] Example 4

[0095] like Figure 3 As shown, this invention also provides a customer service dialogue user emotion recognition system based on multi-dimensional features, which utilizes the aforementioned multi-dimensional feature-based customer service dialogue user emotion recognition method for evaluation. Depending on the implemented functions, the multi-dimensional feature-based customer service dialogue user emotion recognition system 300 may include a data acquisition unit 310, a feature extraction unit 320, a weighted fusion unit 330, and a level classification unit 340. The unit in this invention can also be called a module, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, stored in the memory of the electronic device.

[0096] In this embodiment, the functions of each module / unit are as follows: The data acquisition unit 310 is used to collect multi-source data in real time during customer service dialogue and to preprocess the multi-source data; wherein, the multi-source data includes user text stream, non-text tone signals and interactive behavior time sequence data; The feature extraction unit 320 is used to extract features from the preprocessed multi-source data to obtain multi-dimensional standardized features including text semantic features, tone features and interactive behavior features. The weighted fusion unit 330 is used to perform weighted fusion of the multi-dimensional standardized features based on a pre-built cross-dimensional attention dynamic fusion model, using a product-based nonlinear fusion algorithm, to generate a comprehensive sentiment score; wherein the product-based nonlinear fusion formula is: F = min (S) text ×(1+α S tone )×(1+β S βehavior ),100) Among them, S text For text semantic features, S tone As a feature of tone, S βehaviorFor interactive behavior features, α and β are the adjustment coefficients of the cross-dimensional attention dynamic fusion model, and their values ​​are both in the range of [0,2]. The rating unit 340 is used to determine the emotional level of the comprehensive emotional score according to the preset emotional rating rating standard, and to execute the graded response strategy corresponding to the emotional level; wherein, the preset emotional rating rating standard divides the user's comprehensive emotions into three levels: low risk, medium risk, and high risk.

[0097] More specific implementations of the above-mentioned customer service dialogue user emotion recognition system based on multi-dimensional features can be found in the foregoing description of the embodiments of the customer service dialogue user emotion recognition method based on multi-dimensional features, and will not be detailed here.

[0098] Example 5

[0099] like Figure 4 As shown, the present invention also provides an electronic device 1 for a customer service dialogue user emotion recognition method based on multi-dimensional features.

[0100] The electronic device 1 may include a processor 10, a memory 11, and a bus. It may also include a computer program stored in the memory 11 and executable on the processor 10, such as a multi-dimensional feature-based customer service dialogue user emotion recognition program 12. The memory 11 may include both internal storage units for the multi-dimensional feature-based customer service dialogue user emotion recognition system and external storage devices. The memory 11 can be used not only to store application software and various types of data, such as the code for the multi-dimensional feature-based customer service dialogue user emotion recognition program, but also to temporarily store data that has been output or will be output.

[0101] The memory 11 includes at least one type of readable storage medium. In some embodiments, the memory 11 may be an internal storage unit of the electronic device 1, and in other embodiments, it may be an external storage device of the electronic device 1. Furthermore, the memory 11 may include both internal and external storage units of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as code for a customer service dialogue user emotion recognition program based on multi-dimensional features, but also to temporarily store data that has been output or will be output.

[0102] In some embodiments, the processor 10 may be composed of integrated circuits, or it may be composed of multiple integrated circuits packaged with the same or different functions. The processor 10 is the control core of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes various functions of the electronic device 1 and processes data by running or executing programs or modules stored in the memory 11 (such as a customer service dialogue user emotion recognition program based on multi-dimensional features) and calling data stored in the memory 11.

[0103] The bus can be a peripheral component interconnection standard bus or an extended industry standard structure bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0104] Figure 4 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 4 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0105] The customer service dialogue user emotion recognition program 12 based on multi-dimensional features stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can implement the steps in the customer service dialogue user emotion recognition method based on multi-dimensional features as described above.

[0106] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here. Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0107] This invention also provides a computer-readable storage medium, which may be non-volatile or volatile, storing a computer program. When executed by a processor, the computer program implements the various steps of the customer service dialogue user emotion recognition method based on multi-dimensional features as described above.

[0108] Specifically, the specific implementation method of the computer program when executed by the processor can be referred to the description of the relevant steps in the embodiment of the customer service dialogue user emotion recognition method based on multi-dimensional features, and will not be repeated here.

[0109] In the several embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0110] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0112] Those skilled in the art should understand that various improvements can be made to the customer service dialogue user emotion recognition method and system based on multi-dimensional features proposed in this invention without departing from the scope of this invention. Therefore, any changes, modifications, substitutions, integrations, and parameter changes to these embodiments within the spirit and principles of this invention, achieved through conventional substitutions or by achieving the same function without departing from the principles and spirit of this invention, fall within the protection scope of this invention.

Claims

1. A method for recognizing user emotions in customer service dialogues based on multi-dimensional features, applied to electronic devices, characterized in that: Includes the following steps: S1: Collect multi-source data during customer service dialogue in real time and preprocess the multi-source data; wherein, the multi-source data includes user text stream, non-text tone signals and interaction behavior time sequence data; S2: Extract features from the preprocessed multi-source data to obtain multi-dimensional standardized features, including text semantic features, tone features, and interactive behavior features; S3: Based on a pre-built cross-dimensional attention dynamic fusion model, a weighted fusion of the multi-dimensional standardized features is performed using a product-based nonlinear fusion algorithm to generate a comprehensive sentiment score; wherein, the product-based nonlinear fusion formula is: F=min (S text ×(1+a S tone )×(1+β S behavior ),100) Among them, S text For text semantic features, S tone As a feature of tone, S behavior For interactive behavior features, α and β are the adjustment coefficients of the cross-dimensional attention dynamic fusion model, and their values ​​are both in the range of [0,2]. S4: Determine the emotional level of the comprehensive emotional score according to the preset emotional level classification standard, and execute the graded response strategy corresponding to the emotional level; wherein, the preset emotional level classification standard divides the user's comprehensive emotions into three levels: low risk, medium risk, and high risk.

2. The customer service dialogue user emotion recognition method based on multi-dimensional features according to claim 1, characterized in that, The non-textual mood signals include consecutive punctuation marks, repeated characters, input rhythm, and character capitalization; The interactive behavior time-series data is customer service scenario-specific behavior data, including interruption behavior when the robot does not reply, clicking to transfer to human agent behavior, short-term continuous communication behavior, long-term silence behavior, and like or dislike feedback behavior.

3. The customer service dialogue user emotion recognition method based on multi-dimensional features according to claim 2, characterized in that, Preprocessing the multi-source data includes: Cleaning process is used to remove noise and invalid content from the multi-source data; Normalization is used to unify the format, precision, and value range of the multi-source data; Quantization processing is used to transform non-textual data and behavioral data in the multi-source data into machine-recognizable structured features.

4. The customer service dialogue user emotion recognition method based on multi-dimensional features according to claim 3, characterized in that, The text semantic features are extracted and normalized to the [0,100] interval using BERT or RoBERTa pre-trained language models; The tone features are extracted and normalized to the [0,1] interval using a pre-defined fully connected network; The interactive behavior features are extracted and normalized to the [0,1] interval using a preset weighted model.

5. The customer service dialogue user emotion recognition method based on multi-dimensional features according to claim 4, characterized in that, The cross-dimensional attention dynamic fusion model maps the multi-dimensional standardized features to a unified dimensional space, calculates the feature correlation degree through a preset Transformer encoder, and dynamically generates adjustment coefficients α and β.

6. The customer service dialogue user emotion recognition method based on multi-dimensional features according to claim 5, characterized in that, The preset emotion level classification criteria and corresponding response strategies are as follows: Low-risk range: F < 30, implement standard customer service response strategy; Medium risk range: 30≤F<60, automatically insert empathic remarks and implement reassurance response strategies; High-risk zone: F≥60, force transfer to human agent and push emotional warning, implement upgraded intervention strategy.

7. The customer service dialogue user emotion recognition method based on multi-dimensional features according to claim 6, characterized in that, When any dimension of the multi-dimensional standardized features is missing or subject to noise interference, the attention weight of the missing dimension is automatically reduced, and compensation calculation is performed based on the remaining dimension features.

8. A customer service dialogue user emotion recognition system based on multi-dimensional features, characterized in that, The system includes: The data acquisition unit is used to collect multi-source data in real time during customer service dialogues and to preprocess the multi-source data; wherein, the multi-source data includes user text streams, non-text tone signals, and interactive behavior time-series data; The feature extraction unit is used to extract features from the preprocessed multi-source data to obtain multi-dimensional standardized features, including text semantic features, tone features, and interactive behavior features. The weighted fusion unit is used to perform weighted fusion of the multi-dimensional standardized features based on a pre-built cross-dimensional attention dynamic fusion model, using a product-based nonlinear fusion algorithm to generate a comprehensive sentiment score; wherein the product-based nonlinear fusion formula is: F=min (S text ×(1+a S tone )×(1+β S behavior ),100) Among them, S text For text semantic features, S tone As a feature of tone, S behavior For interactive behavior features, α and β are the adjustment coefficients of the cross-dimensional attention dynamic fusion model, and their values ​​are both in the range of [0,2]. The rating unit is used to determine the emotional level of the comprehensive emotional score according to a preset emotional rating standard, and to execute a graded response strategy corresponding to the emotional level; wherein the preset emotional rating standard divides the user's comprehensive emotions into three levels: low risk, medium risk, and high risk.

9. An electronic device, characterized in that, The system includes a memory, a processor, and a multi-dimensional feature-based customer service dialogue user emotion recognition program stored in the memory and executable on the processor. When executed by the processor, the multi-dimensional feature-based customer service dialogue user emotion recognition program implements the multi-dimensional feature-based customer service dialogue user emotion recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the customer service dialogue user emotion recognition method based on multi-dimensional features as described in any one of claims 1 to 7.

Images