A method, system, and medium for agent assistance based on multi-dimensional risk assessment and generative optimization

Abstract

This application provides a method, system, and medium for agent assistance based on multidimensional risk assessment and generative optimization, belonging to the field of artificial intelligence technology. The method includes: acquiring interaction data and parsing the time series of emotional state values; calculating emotional change trends; calculating a multidimensional communication risk level by combining current emotions, trends, and user profiles to trigger script recommendations; generating scripts by matching from a knowledge base or calling a generative model based on the risk level; and monitoring the script application effect as feedback data to update the knowledge base score or fine-tune the generation model. This application solves the problem of existing technologies lacking dynamic consideration and closed-loop optimization capabilities, providing more accurate and contextualized assistance, and enabling the system's adaptive evolution.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a seat assistance method, system and medium based on multidimensional risk assessment and generative optimization. Background Technology

[0002] In customer service sectors such as call centers and online customer service, the service quality of agents directly impacts customer satisfaction and corporate image. Traditional agent support systems are typically passive, primarily providing a knowledge base for agents to manually search. This model heavily relies on agents' personal experience and on-the-spot judgment. When faced with emotionally charged or complex customers, agents often struggle to quickly and accurately identify the most appropriate communication strategy, leading to low communication efficiency and potentially escalating negative customer emotions.

[0003] To address these issues, some existing technologies attempt to incorporate emotion monitoring functionality. For example, some systems can analyze and display the trajectory of a user's emotional state over time, providing suggestions to agents when the emotional state meets preset intervention criteria. However, these existing technologies still have significant shortcomings: First, their judgment on the timing of intervention is often based solely on a static snapshot of the current emotion or simple rules, lacking consideration of dynamic emotional trends. They cannot distinguish between a customer calming down from anger and one whose anger is rapidly escalating, potentially leading to inappropriate advice. Second, these systems lack a comprehensive quantitative assessment of communication risks, failing to integrate information from multiple dimensions such as emotional trends and user history profiles for accurate decision-making. Third, their recommended scripts are usually derived from a fixed knowledge base, appearing rigid and lacking flexibility and personalization when facing complex or non-standard scenarios. Most importantly, existing technologies generally lack effective closed-loop feedback and self-optimization capabilities. The system cannot learn from the application effects of scripts and continuously iterate its recommendation strategies, resulting in a rigid script library that cannot adapt to constantly changing business scenarios, making it difficult to improve overall assistance effectiveness. Summary of the Invention

[0004] The purpose of this application is to address the problems in the prior art where existing agent assistance systems rely solely on static emotion snapshots for recommendations, lack consideration for dynamic changes in emotions, and do not possess closed-loop feedback optimization capabilities, resulting in poor assistance effects and an inability to adaptively improve. This application provides an agent assistance method, system, and medium based on multidimensional risk assessment and generative optimization.

[0005] To achieve the above objectives, this application provides a method for agent assistance based on multidimensional risk assessment and generative optimization. The method includes: acquiring preset rules, pre-stored user profile information, and interactive session data between the user and the agent; periodically parsing the user's emotional state values ​​based on the interactive session data to form a time series of emotional state values, and extracting the session context; calculating the user's emotional change trend based on the time series of emotional state values; calculating a multidimensional communication risk level by combining the current emotional state value in the time series of emotional state values, the emotional change trend, and the pre-stored user profile information; and triggering script recommendation when the multidimensional communication risk level exceeds a first preset threshold; performing operations such as matching scripts from a script knowledge base or generating scripts by calling a generative language model, according to preset rules, the multidimensional communication risk level at the time of triggering, and the session context, to obtain recommended scripts; monitoring the user's emotional change after pushing the recommended scripts as feedback data, and updating the effectiveness score of the scripts in the script knowledge base and / or fine-tuning the generative language model based on the feedback data.

[0006] Optionally, the interactive session data includes voice data and text data transcribed from the voice data; the step of periodically parsing the user's emotional state value includes: using multimodal data fusion technology to fuse voice analysis results based on the acoustic features of the voice data and text analysis results based on the semantics of the text data.

[0007] Optionally, the multimodal data fusion technology includes: using a convolutional neural network to analyze the acoustic features of speech data to obtain acoustic feature analysis results; using a Transformer model to perform semantic analysis on text data to obtain semantic analysis results; and using an attention fusion layer to perform weighted fusion of the acoustic feature analysis results and the semantic analysis results.

[0008] Optionally, the step of calculating the multidimensional communication risk level specifically involves: calculating the multidimensional communication risk level using a weighted summation model.

[0009] Optionally, the inputs to the weighted summation model include: a normalized current emotional state value, a normalized emotional change trend, and a risk bias term based on the user profile.

[0010] Optionally, the operation of matching dialogue from the dialogue knowledge base or generating dialogue by calling the generative language model includes: under the condition that the multidimensional communication risk level exceeds a second preset threshold, or the profile tags in the pre-stored user profile information and the conversation context meet preset matching conditions, calling the generative language model to generate dialogue; otherwise, matching recommended dialogue from the dialogue knowledge base; wherein the second preset threshold is greater than or equal to the first preset threshold.

[0011] Optionally, calculating the user's emotional change trend includes: calculating the linear regression slope of the emotional state value within a preset time window, and using the linear regression slope as the emotional change trend.

[0012] Optionally, updating the effectiveness score of the dialogue in the dialogue knowledge base and / or fine-tuning the generative language model based on the feedback data further includes: using recommended dialogue generated by the generative language model and determined to be effective as high-quality samples for further fine-tuning the generative language model. Here, being determined to be effective means that within a preset time period after the recommended dialogue is pushed, the user's current emotional state value increases by more than a preset emotional threshold, or the multidimensional communication risk level decreases by more than a preset risk threshold.

[0013] To achieve the above objectives, this application also provides a seat assistance system based on multidimensional risk assessment and generative optimization. The system includes: a data acquisition module for acquiring interactive session data between the user and the agent; a sentiment analysis engine connected to the data acquisition module for periodically parsing the user's sentiment state values ​​based on the interactive session data to form a time series of sentiment state values, calculating the user's sentiment change trend based on the time series of sentiment state values, and extracting the session context; and a multidimensional decision engine connected to the sentiment analysis engine for comprehensively calculating multidimensional communication risks, etc., by integrating the current sentiment state value in the time series of sentiment state values, the sentiment change trend, and pre-stored user profile information. The system comprises: a script management and recommendation module, including a script knowledge base and a generative language model, connected to the multidimensional decision engine; a script management and recommendation module, which is triggered when the multidimensional communication risk level exceeds a first preset threshold, and performs operations such as matching scripts from the script knowledge base or generating scripts by calling the generative language model according to preset rules, the multidimensional communication risk level, and the conversation context; and a closed-loop optimization module, connected to the emotion analysis engine and the script management and recommendation module, which monitors changes in user emotions after pushing scripts as feedback data, and updates the effectiveness score of the script knowledge base in the script management and recommendation module and / or fine-tunes the generative language model based on the feedback data.

[0014] To achieve the above objectives, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the preceding claims.

[0015] Compared with existing technologies, this application has the following beneficial effects: First, by introducing the analysis of emotion change trends and combining it with a quantitative multidimensional risk assessment model, this application can predict and accurately measure the risk trajectory of a conversation, providing more forward-looking and targeted assistance, and improving the accuracy and foresight of the assistance. Second, by introducing a generative language model, this application can break through the limitations of a fixed script library, providing highly contextualized and personalized dynamic scripts that go beyond templates in complex scenarios, achieving highly contextualized intelligent responses. Third, through dual closed-loop feedback fine-tuning of knowledge base scoring and generative models, the system can learn from each interaction, automatically optimize script strategies, achieve self-iteration, reduce manual maintenance costs, and thus realize the system's adaptive evolution capability. Finally, precise and intelligent system assistance can effectively help agents cope with high-pressure communication scenarios, reduce psychological pressure, shorten the growth cycle of new agents, and thus reduce the workload and training costs of agents. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the seat assistance method provided in an embodiment of this application.

[0017] Figure 2 The system signaling interaction timing diagram provided for the embodiments of this application.

[0018] Figure 3 This is a schematic diagram of a multidimensional communication risk assessment model provided in an embodiment of this application.

[0019] The main reference numerals in the attached diagrams are explained as follows: S101: Acquire interaction data; S102: Multimodal sentiment analysis; S103: Calculate sentiment change trends; S104: Acquire user profiles; S105: Quantitatively assess multidimensional communication risks; S106: Determine if the risk exceeds the threshold; S107: Determine whether to invoke the generation model; S108: Match recommended scripts; S109: Generate recommended scripts; S110: Push scripts to agents; S111: Monitor the application effect of scripts; S112: Update the script effectiveness scoring / fine-tune the model. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and specific embodiments. It is understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0021] Example 1 This embodiment provides a basic framework for an agent assistance method and system based on multidimensional risk assessment and generative optimization. This framework aims to address issues such as inaccurate timing and rigid scripts in existing technologies by dynamically sensing user emotional trends, combining multidimensional information for risk assessment, triggering script recommendations, and constructing a closed-loop optimization mechanism.

[0022] The system can be deployed on a cloud server or a local server cluster. Its core processing server mainly includes multiple functional modules implemented by software programs. These modules work together to complete the entire process of agent assistance.

[0023] Specifically, the system includes: a data acquisition module, a sentiment analysis engine, a multi-dimensional decision engine, a script management and recommendation module, a closed-loop optimization module, and an agent interaction module.

[0024] The data acquisition module, serving as the entry point for system interaction with the outside world, is responsible for acquiring real-time interactive session data between users and agents. In a call center scenario, this data primarily consists of two-way voice streams. In one embodiment of this application, this module can capture real-time audio data through an interface protocol with the call center telephone system (e.g., a session initiation protocol or computer telephony integration technology), and stream the acquired data to the sentiment analysis engine for processing.

[0025] The sentiment analysis engine, as the core component for emotion state perception, is responsible for receiving and processing interactive session data from the data acquisition module. Its main task is to periodically parse the user's emotion state values, thus forming a time series of emotion state values. For example, the engine can slice and analyze the user's speech stream at a preset time frequency (e.g., once per second), outputting a quantified emotion state value. This value can be a continuous value in the range [-1, 1], where -1 represents extremely negative emotions (e.g., anger, despair), 0 represents neutral emotions, and 1 represents extremely positive emotions (e.g., happiness, satisfaction). To facilitate subsequent trend calculations, the sentiment analysis engine caches all emotion state values ​​within a historical time period (e.g., the past 60 seconds), forming time series data. Furthermore, the engine is also responsible for real-time speech recognition of the speech data, converting it into text data to extract session context information for use by subsequent modules.

[0026] The multi-dimensional decision engine, as the core of the entire auxiliary system, is responsible for determining the appropriate timing for providing assistance. This engine connects to the sentiment analysis engine to receive its output of sentiment state value time series and conversation context. Simultaneously, it connects to an external user profile database to obtain relevant information about the current user. It should be noted that the user profile database can pre-store user historical interaction records, customer value levels, complaint history, preferences, and other tag information. The core function of the multi-dimensional decision engine lies in integrating information from three dimensions: first, the current sentiment state value provided by the sentiment analysis engine; second, the sentiment change trend calculated based on the sentiment state value time series; and third, user profile information obtained from the user profile database. Through a preset quantitative model, the engine calculates a real-time multi-dimensional communication risk level. When this risk level exceeds a preset trigger threshold (i.e., the first preset threshold), the multi-dimensional decision engine triggers the script recommendation process and notifies the script management and recommendation module.

[0027] The script management and recommendation module, triggered by the multi-dimensional decision engine, is responsible for generating or selecting suitable recommended scripts. This module internally includes a script knowledge base and a generative language model. This basic embodiment primarily describes the application of the script knowledge base. The script knowledge base stores a large number of structured script entries, each associated with a specific contextual label (such as the business type "refund" and the risk level "high risk") and its own validity score. Upon receiving a trigger signal, the script management and recommendation module searches the script knowledge base based on the risk level and conversation context transmitted by the multi-dimensional decision engine (e.g., the keyword "logistics" extracted from the transcribed text). If multiple scripts are matched, the script with the highest validity score is selected as the final recommended script.

[0028] The agent interaction module, serving as the interface between the system and agents, is responsible for receiving recommended scripts determined by the script management and recommendation module and pushing them to the agent's interface in a clear and non-intrusive manner. For example, a window can pop up on the agent's computer screen displaying the recommended script text for the agent's reference or direct use.

[0029] The closed-loop optimization module is key to the system's self-evolution. This module connects to both the sentiment analysis engine and the script management and recommendation module. Its workflow begins after a script is pushed to the system. The closed-loop optimization module continuously monitors changes in the user's emotional state within a preset time period (e.g., 90 seconds) after the script is pushed, using these changes as feedback data on the script's effectiveness. Based on this feedback data, the closed-loop optimization module evaluates the effectiveness of the recommended scripts. If the evaluation result is positive (e.g., the user's mood improves significantly), it sends a command to the script management and recommendation module to update the effectiveness score of the corresponding script in the specific context in the script knowledge base, for example, increasing the score by 0.1 points. Conversely, if the effect is poor or counterproductive, the score can be lowered accordingly. In this way, the system can automatically filter and strengthen effective scripts while eliminating ineffective ones.

[0030] The following will combine Figure 1 and Figure 2 This document provides a clearer and more complete description of the specific flow and interaction process of the agent assistance method in this embodiment. Figure 1 This is a flowchart illustrating the method. Figure 2 This is a timing diagram of the system signaling interaction. (Example:) Figure 2 As shown, system interaction mainly takes place between user terminals, backend servers, and agent terminals; user terminals and backend servers continuously transmit interactive data streams; the backend server internally executes a continuous analysis loop, constantly generating sentiment analysis requests and results; during the risk assessment phase, when the assessment result meets the preset risk exceeding threshold condition (alt), the backend server pushes a script recommendation message to the agent terminal; subsequently, based on the acquired effect evaluation data stream, the backend server generates model fine-tuning instructions for optimization. Figure 1 The detailed process is as follows: The process begins with step S101: acquiring interaction data. After the user establishes a call with the agent, the data acquisition module starts working, capturing the two-way voice data stream in real time and sending it to the backend server for processing.

[0031] Subsequently, in steps S102 and S103, the emotion analysis engine performs processing. In step S102, the engine performs multimodal emotion analysis on the received user voice data (in this basic embodiment, it mainly focuses on acoustic feature analysis), periodically (e.g., once per second) generating the current emotion state value, and aggregating these values ​​to form a time series. Simultaneously, the speech is transcribed into text for extracting the conversation context. In step S103, the engine calculates the emotion change trend based on this time series. As a specific calculation method, linear regression analysis can be performed on the emotion state values ​​within a preset time window (e.g., the most recent 30 seconds), and the slope of the resulting regression line is used as a quantitative indicator of the emotion change trend. A negative slope indicates that the emotion is deteriorating, a positive slope indicates that the emotion is improving, and the absolute value of the slope represents the degree of drastic change.

[0032] Next, the process proceeds to steps S104 and S105, executed by the multi-dimensional decision engine. In step S104, the engine retrieves the user's profile information from the user profile database based on the current user's identity identifier (such as a phone number). In step S105, the engine performs a quantitative assessment of the multi-dimensional communication risk. In this embodiment, a simple rule-based model can be used. For example, the rule could be set as follows: "When the current emotional state value is less than -0.5 (indicating negative emotions) and the emotional change trend is less than -0.01 (indicating that the emotions are still deteriorating), it is judged as a high-risk event."

[0033] In step S106, the multidimensional decision engine determines whether the calculated risk level exceeds a preset trigger threshold (i.e., the first preset threshold). If it does not exceed the threshold, it indicates that the current communication risk is controllable, and the system returns to step S101 to continue monitoring without interfering with the agent. If it exceeds the threshold, it indicates that intervention is required, and the process proceeds to the next step, namely step S107: determining whether to invoke the generative model.

[0034] In this embodiment, as a basic implementation, assuming the conditions for calling the generation model are not currently met, the process proceeds to step S108: matching recommended scripts (if the calling conditions are met, the process proceeds to step S109: generating recommended scripts; the specific judgment conditions and generation logic will be detailed in subsequent embodiments). Taking step S108 as an example, the script management and recommendation module is triggered. Based on the current risk level (high risk) and the conversation context (e.g., keywords "refund" and "logistics"), it searches the script knowledge base. For example, the module will look for scripts tagged with both "high risk" and "refund," and select the one with the highest effectiveness score, such as, "Sir / Madam, I fully understand your anxiety about the refund issue. Please rest assured, I will do my best to help you resolve this problem."

[0035] After obtaining the recommended script, in step S110, the agent interaction module pushes the script to the agent's operation interface.

[0036] Finally, the process enters the closed-loop optimization phase executed by the closed-loop optimization module, namely steps S111 and S112. In step S111, the module begins monitoring the effectiveness of the script application, continuously obtaining the user's emotional state value from the sentiment analysis engine within a certain period (e.g., 90 seconds) after the script is pushed. In step S112, the module updates the script effectiveness score based on the monitoring results. For example, the module can calculate the average user's emotional value over these 90 seconds and compare it with the average emotional value in the 30 seconds before the script was pushed. If the former is higher than the latter by more than a preset emotional threshold (e.g., 0.2), the recommendation is considered effective, and the effectiveness score of the script in the "high-risk + refund" scenario is increased by 0.1. After the update is completed, a complete closed loop of the system process ends, and the system returns to step S101 to prepare for the next interaction.

[0037] Through the above process, this embodiment constructs a seat assistance system that can proactively assist based on emotional dynamics and achieve self-optimization through effect feedback, thereby effectively improving the accuracy of assistance and the long-term effectiveness of the system.

[0038] Example 2 This embodiment, based on Embodiment 1, focuses on optimizing the multidimensional decision engine, aiming to provide a more refined and quantitative multidimensional communication risk assessment model. Compared to the rule-based decision-making method in Embodiment 1, the weighted summation model introduced in this embodiment makes the risk assessment process more dynamic and accurate, in order to cope with certain complex scenarios.

[0039] The overall system structure is basically the same as in Example 1; the core difference lies in the implementation logic within the multi-dimensional decision engine. Please refer to [link / reference]. Figure 3 This figure is a schematic diagram of the multidimensional communication risk assessment model provided in this embodiment.

[0040] like Figure 3 As shown, the multi-dimensional decision engine 30 receives three main inputs: current emotional state input 31, emotional change trend input 32, and user profile input 33. Internally, the engine uses a weighted summation mathematical model to fuse the information from these three dimensions into a single, continuous communication risk level output 34. This risk level is then compared with a preset threshold to determine whether to trigger script recommendations.

[0041] The specific calculation process is as follows: First, the inputs are normalized so that weighted calculations can be performed on the same scale.

[0042] 1. Current Emotional State Input: This input is the current emotional state value output by the emotion analysis engine, denoted as... Its range is [-1, 1]. To convert it into a risk measure, the following transformation is performed to obtain the normalized current emotional state risk term. : Through this transformation, when emotions are extremely positive ( )hour, When emotions are neutral ( )hour, When emotions are extremely negative ( )hour, .so, The value range is [0, 1], and the larger the value, the higher the emotional risk.

[0043] 2. Sentiment Change Trend Input: This input is the slope of the sentiment change trend calculated by the sentiment analysis engine, denoted as... The slope can be any real number. To normalize it to the [0, 1] interval and make its larger value represent higher risk (i.e., faster mood deterioration), a sigmoid function (such as the sigmoid function) can be used for mapping. For example, the normalized mood change trend risk term. It can be calculated in the following ways: in, This is an adjustable parameter used to control the sensitivity of the slope. When the slope... When it is a large negative number (indicating a rapid deterioration of mood), Approaching 1; when the slope is 0 (emotional stability), The value is 0.5; when the slope is a large positive number (indicating rapid improvement in mood), Approaching 0.

[0044] 3. User Profile Input: This input is a risk bias item generated based on user tags obtained from the user profile database. The system can pre-set risk scores for different user profile tags. For example, the risk score for the "complaint history" tag is +0.3, the risk score for the "important customer" or "VIP customer" tag is -0.1 (indicating that more careful handling is required and the tolerance is lower, but it does not necessarily represent a risk), and the risk score for the "first caller" tag is +0.1 (due to higher uncertainty). It can be the sum or weighted sum of multiple label scores, and it also needs to be normalized to a specific range, such as [-0.5, 0.5].

[0045] After obtaining the three normalized risk items above, the multidimensional decision engine uses a weighted summation formula to calculate the final communication risk level R. Combined with... Figure 3 As shown, Figure 3For the sake of simplicity, the calculation logic shown in the diagram is R = w1. E +w2 T + w3 P, where Figure 3 The variables E, T, and P in the text are equivalent to the normalized current emotional state risk term in the main text. Normalized sentiment change trend risk item and user profile risk bias items The complete calculation formula is as follows: .

[0046] Among them, w1, w2, and w3 are preset weight coefficients, and their sum is not necessarily 1. They reflect the importance of information in different dimensions in risk assessment.

[0047] Calculated communication risk level It is a continuous numerical value. The multidimensional decision engine compares it with a preset first threshold (e.g., 0.7). When At that time, the script recommendation process is triggered.

[0048] The following is a concrete example demonstrating the application of this model. Suppose a user calls; the system retrieves their phone number from the user profile database and finds the tag "has a history of complaints," along with the corresponding risk bias item. The value is set to 0.3. The sentiment analysis engine continues to operate after the call begins. At some point, the system detects the user's current emotional state value. Slope of the trend of emotion change At this point, the multi-dimensional decision engine begins calculations: 1. Normalize the current emotional risk term: 2. Normalized sentiment trend risk term (assuming it has been mapped using a sigmoid function): 3. User profile risk bias items: 4. Using weights Calculate the overall risk level: In this situation, the communication risk level R reaches the first preset threshold of 0.7, and the system will continue monitoring. A few seconds later, the user's emotions continue to deteriorate, and the current emotional state value... Decreased to -0.7, trend slope Reduced to -0.04. The multidimensional decision engine recalculated: 1. 2. Increase to 0.9. 3. The overall risk level remains unchanged at 0.3. 4. New overall risk level: . at this time, The risk level exceeded the first preset threshold. Accordingly, the multi-dimensional decision engine immediately triggered the script recommendation process and transmitted the current risk level of 0.755 along with the conversation context information to the script management and recommendation module.

[0049] By using this weighted quantification method, the risk assessment model in this embodiment can more sensitively and accurately reflect subtle changes in the communication status, comprehensively considering the immediate state of emotions, development trends, and inherent attributes of users, thereby achieving more accurate decision-making on the timing of assistance, avoiding misjudgments or omissions that may be caused by simple rules, and significantly improving the accuracy and foresight of assistance.

[0050] Example 3 This embodiment significantly enhances the functionality of the dialogue management and recommendation module and the closed-loop optimization module, based on Embodiments 1 and 2. Its core lies in the introduction of a generative language model, enabling a leap from "selecting dialogue" to "generating dialogue," and expanding the closed-loop optimization mechanism to simultaneously optimize both the dialogue knowledge base and the generative language model, thus forming a dual-objective closed-loop optimization system.

[0051] In this embodiment, the internal structure of the script management and recommendation module is more complex. In addition to the structured script knowledge base mentioned in Embodiment 1, it also integrates a generative language model. As an optional implementation, this generative language model can be a smaller version of a large language model that has been fine-tuned and trained using domain-specific (e.g., customer service) corpora, particularly high-quality transcripts of conversations between excellent agents and customers. The purpose of this fine-tuning is to enable the model to master the context, tone, and problem-solving strategies of customer service communication.

[0052] The script management and recommendation module executes a dual-path script strategy based on information from the multi-dimensional decision engine. This decision logic is illustrated in the flowchart. Figure 1 This is reflected in step S107, "Determine whether to call the generative model." The specific rules are as follows: 1. When the multi-dimensional decision engine calculates the communication risk level... If the score exceeds the first preset threshold but is below a higher second preset threshold (e.g., the first threshold is 0.7 and the second threshold is 0.9), or if a highly matched and highly rated script can be found in the script knowledge base for the current scenario, the system will prioritize the "match script from the knowledge base" path (corresponding to...). Figure 1 Step S108). This ensures that, under normal risk scenarios, the system can quickly and efficiently provide validated standardized scripts, guaranteeing response speed and stability. 2. When the communication risk level... Exceeding the second preset threshold (e.g.) In the event of a user profile labeled "technical expert" and the dialogue context involving complex issues that are difficult to cover in the knowledge base, such as "product design flaws," the system will invoke a generative language model to execute the path of "generating recommended dialogue" (corresponding to...). Figure 1 Step S109).

[0053] When the generative language model is invoked, the script management and recommendation module constructs a detailed prompt containing rich contextual information to guide the model in generating high-quality scripts. This prompt may include: the current communication risk level. Current emotional state value and trends in mood changes Key user profile tags, such as "VIP customer," "technical background," and "multiple complaints." A contextual summary of the current conversation or a transcript of recent rounds. A clear task instruction, such as: "Generate a professional, empathetic, and reassuring response that acknowledges the value of the user's feedback and proposes a solution to escalate the issue to the product team." Once the generative language model receives this prompt, it will dynamically generate a personalized, highly contextualized recommendation.

[0054] Correspondingly, the functionality of the closed-loop optimization module has also been expanded, achieving dual-objective optimization. In step S112, "Update the script effectiveness score / fine-tune the model," its specific behavior depends on the source of the recommendation: 1. If the recommended script comes from a script knowledge base, the logic of the closed-loop optimization module is the same as in Example 1, that is, it dynamically updates the effectiveness score of the script in the knowledge base based on feedback data on the script's application effect. 2. If the recommended script is generated by a generative language model, the closed-loop optimization module will perform more complex operations. First, it will also evaluate the effectiveness of the generated script. As a specific criterion: within a preset time period after pushing the script (e.g., 90 seconds), if the user's current emotional state value increases by more than a preset emotional threshold (e.g., an increase of 0.3), or the multidimensional communication risk level... If the risk decrease exceeds a preset risk threshold (e.g., a decrease of 0.2), and is deemed effective, the closed-loop optimization module stores this successful interaction case (including the prompts used by the generation model, the generated dialogue, and the final positive effect) as a high-quality training sample in a dedicated database. Once a sufficient number of samples have been accumulated, the system can use these samples to further fine-tune the generative language model. This allows the model to learn from real-world successes and continuously improve its ability to generate effective dialogue in specific scenarios. Furthermore, for generated dialogue that has proven highly effective, the system can semi-automatically or manually review it and convert it into a new structured entry, adding it to the dialogue knowledge base to enrich its content.

[0055] The following is a detailed explanation of a workflow. A user, a seasoned user of a certain software, complained over the phone about the unreasonable logic design of a certain advanced feature (Feature A), which was hindering his workflow. 1. The sentiment analysis engine detected negative sentiment from the user (…). ), and it is still worsening ( 2. The multi-dimensional decision engine combines the "technical expert" tag in the user profile to calculate the communication risk level. 3. The risk level exceeds the second preset threshold of 0.9. Therefore, in step S107, the system decides to invoke the generative model. 4. The script management and recommendation module sends a prompt to the generative language model, containing: "Risk level 0.95, Emotion -0.8, Trend -0.05, Profile - Technical Expert, Context - Complaint: Function A's logic is unreasonable. Please generate professional, empathetic reassurance scripts and propose constructive solutions." 5. In step S109, the generative language model generates the script: "Thank you very much for your professional feedback. We highly value the voice of experts. The logic problem you mentioned regarding Function A is indeed worth exploring in depth, and this is exactly the valuable opinion we hope to get from users. Please allow me to record your suggestion verbatim and submit it directly to the core members of our product development team for a specialized evaluation. Could you please take a moment to explain in detail what your expected logic is?" 6. In step S110, the script is pushed to the agent. After the agent adopts and uses it, it can be monitored that the user's emotions are greatly alleviated, and a constructive discussion with the agent begins. 7. In step S111, the closed-loop optimization module detects that within 90 seconds after the script is pushed, the user's emotional state value recovers from -0.8 to -0.2, an increase of 0.6, far exceeding the effectiveness threshold of 0.3. 8. In step S112, the closed-loop optimization module packages the complete data of this interaction (prompt, generated script, effect) into a high-quality training sample and stores it in the training database.

[0056] Through the solution in this embodiment, the system can not only provide stable and efficient assistance in conventional scenarios, but also offer personalized dialogue that transcends fixed templates and is rich in creativity and empathy in complex, high-risk, and non-standard scenarios. More importantly, through closed-loop fine-tuning of the generative model, a self-learning and self-evolving system is constructed, whose problem-solving capabilities continuously improve with usage time.

[0057] Example 4 This embodiment focuses on optimizing the sentiment analysis engine, aiming to improve the accuracy and robustness of sentiment recognition by introducing multimodal data fusion technology. In many modern customer service scenarios, interactions are not limited to voice but may also include text chat, or voice calls may be transcribed into text in real time. This embodiment utilizes this type of multimodal data to gain a more comprehensive understanding of the user's emotional state.

[0058] In this embodiment, the internal structure of the sentiment analysis engine is more advanced than that described in Embodiment 1, and it is suitable for scenarios that can simultaneously acquire user voice data and real-time text data (which can be transcribed from voice in real time or be chat text directly entered by the user). Its core lies in the fusion of analysis results from two or more different models.

[0059] In practice, a sentiment analysis engine can include the following sub-parts: 1. Speech Analysis Sub-model: This model is specifically designed for processing speech data and can be based on a convolutional neural network (CNN) architecture. The input to this model is not the raw audio waveform, but rather extracted acoustic features such as pitch, speech rate, volume, and Mel-frequency cepstral coefficients. CNNs excel at capturing local patterns and structures that reflect emotions from these temporal features. For example, high pitch, fast speech rate, and large volume fluctuations are often associated with excitement or anger. This sub-model outputs a preliminary emotion state vector based on the acoustic features.

[0060] 2. Text Analysis Sub-model: This model processes text data transcribed in real-time from speech or directly input by the user. As an optional implementation, it can employ state-of-the-art pre-trained language models based on transformer architectures (such as BERT or its variants). This model is capable of deeply understanding the semantic content, emotional tone, and underlying intent of the text. For example, the presence of words like "unbelievable" or "extremely disappointed" in the text directly indicates negative emotions. This sub-model outputs a preliminary, semantically based emotion state vector.

[0061] 3. Attention Fusion Layer: This is crucial for achieving multimodal fusion. In real-world communication, the emotional information carried by speech and text is sometimes consistent, and sometimes inconsistent or even contradictory. For example, a person might say "It's okay" but their tone is very cold and impatient. To address this, this embodiment introduces an attention fusion layer. This layer receives two preliminary emotion vectors from the speech analysis sub-model and the text analysis sub-model as input and dynamically calculates the relative importance (i.e., attention weight) of the speech modality and the text modality in the current context. For example, when the text content is neutral (e.g., "Okay, I understand") but the acoustic features are high-pitched and rapid, the attention fusion layer automatically assigns a higher weight to the speech analysis result. Conversely, if the user's speech tone is flat, but the text is full of strong negative words, the text analysis result will have a higher weight.

[0062] The fusion process can be represented as: in, and These are the emotion vectors for speech and text, respectively. The weights are dynamically calculated by the attention mechanism based on the current input, and their values ​​are between 0 and 1.

[0063] Final emotional state value output: After processing by the attention fusion layer, a fused emotional state vector is obtained. Finally, this fusion vector is mapped to a continuous emotional state value in the range [-1, 1] defined in Example 1 via a fully connected layer or a simple linear transformation. This value is then passed to the multidimensional decision engine for subsequent risk assessment.

[0064] The advantages of this embodiment are demonstrated below through a workflow. A user is communicating with an agent through an online chat system that supports voice input. 1. The user inputs a sentence via voice, which the system recognizes as the text "OK, I understand," while retaining the original voice segment. 2. In step S102, the sentiment analysis engine begins operation. The text analysis sub-model analyzes the text "OK, I understand," and due to its neutral content, its output sentiment vector is biased towards neutrality (close to 0). The voice analysis sub-model analyzes the corresponding voice segment and detects that its pitch is significantly higher than the user's baseline pitch, and the speech rate is also 30% faster than usual. These acoustic features indicate that the user may be in a state of impatience or anxiety. Therefore, the voice analysis sub-model outputs a vector pointing to negative emotion (e.g., equivalent to -0.6). The attention fusion layer receives these two significantly different sentiment vectors. Due to the conflict between the text content and the emotion expressed by the acoustic features, the attention mechanism is activated. It assigns higher weights to the voice analysis result according to a preset fusion strategy (e.g., placing more trust in non-verbal signals in cases of conflict), for example... The final calculated emotional state value after fusion is as follows: 3. The sentiment analysis engine ultimately outputs a current sentiment state value of -0.48 to the multi-dimensional decision engine. 4. Understandably, if unimodal text analysis is used, the system might misjudge the user's emotion as neutral, thus missing the optimal intervention opportunity. This embodiment, however, accurately captures the user's true emotions hidden behind the text through multimodal fusion. Even if the sentiment value itself has not yet reached the trigger threshold, it provides a more accurate starting point for subsequent calculations of sentiment change trends, making the entire system's perception of the communication atmosphere more sensitive and reliable.

[0065] By integrating speech acoustic features and text semantic information, this embodiment significantly improves the accuracy of emotion recognition, especially in effectively identifying "disguised" or complex situations where the text does not match the true emotion. This provides higher-quality input for subsequent risk assessment and decision-making, which forms a solid foundation for improving the overall performance of the agent assistance system based on multidimensional risk assessment and generative optimization.

[0066] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A seat assistance method based on multidimensional risk assessment and generative optimization, characterized in that, include: The system acquires preset rules, pre-stored user profile information, and interactive session data between users and agents. Based on the interactive session data, it periodically parses out the user's emotional state values ​​to form a time series of emotional state values ​​and extracts the session context. Based on the time series of the emotional state values, calculate the user's emotional change trend; By combining the current emotional state value in the time series of the emotional state value, the emotional change trend, and the pre-stored user profile information, a multidimensional communication risk level is calculated, and when the multidimensional communication risk level exceeds a first preset threshold, a script recommendation is triggered. Based on preset rules, the multidimensional communication risk level at the time of triggering, and the conversation context, the operation of matching the script from the script knowledge base or calling the generative language model to generate the script is performed to obtain the recommended script; The system monitors changes in user emotions after the recommended dialogue is pushed as feedback data, and updates the effectiveness score of the dialogue in the dialogue knowledge base and / or fine-tunes the generative language model based on the feedback data.

2. The method according to claim 1, characterized in that, The interactive session data includes voice data and text data transcribed from the voice data; The step of periodically parsing the user's emotional state value includes: Multimodal data fusion technology is employed to integrate speech analysis results based on acoustic features of the speech data and text analysis results based on semantics of the text data.

3. The method according to claim 2, characterized in that, The multimodal data fusion technology includes: The acoustic features of speech data are analyzed using a convolutional neural network to obtain the acoustic feature analysis results; The Transformer model is used to perform semantic analysis on text data, and the semantic analysis results are obtained; and The acoustic feature analysis results and the semantic analysis results are weighted and fused using an attention fusion layer.

4. The method according to claim 1, characterized in that, The steps for calculating the risk level of multidimensional communication are as follows: The multidimensional communication risk level is calculated using a weighted summation model.

5. The method according to claim 4, characterized in that, The inputs to the weighted summation model include: Normalized current emotional state value, normalized emotional change trend, and risk bias based on user profile.

6. The method according to claim 1, characterized in that, The operation of matching dialogue from a dialogue knowledge base or generating dialogue by calling a generative language model includes: When the multidimensional communication risk level exceeds the second preset threshold, or when the profile tags in the pre-stored user profile information meet the preset matching conditions with the conversation context, the generative language model is invoked to generate the dialogue. Otherwise, a recommended script is matched from the script knowledge base; Wherein, the second preset threshold is greater than or equal to the first preset threshold.

7. The method according to claim 1, characterized in that, The calculation of the user's emotional change trend includes: Calculate the linear regression slope of the emotional state value within a preset time window, and use the linear regression slope as the trend of the emotional change.

8. The method according to claim 1, characterized in that, The step of updating the effectiveness score of the dialogue in the dialogue knowledge base and / or fine-tuning the generative language model based on the feedback data further includes: The recommended phrases generated by the generative language model and deemed valid are used as high-quality samples for further fine-tuning and training of the generative language model. Here, being deemed valid means that within a preset time period after the recommended phrase is pushed, the user's current emotional state value increases by more than a preset emotional threshold, or the multidimensional communication risk level decreases by more than a preset risk threshold.

9. A seat assistance system based on multidimensional risk assessment and generative optimization, characterized in that, include: The data acquisition module is used to acquire interactive session data between users and agents; The emotion analysis engine, connected to the data acquisition module, is used to periodically parse the user's emotional state value based on the interactive session data to form a time series of emotional state values, calculate the user's emotional change trend based on the time series of emotional state values, and extract the session context. A multi-dimensional decision engine, connected to the emotion analysis engine, is used to calculate the multi-dimensional communication risk level by integrating the current emotion state value in the time series of the emotion state value, the emotion change trend, and the pre-stored user profile information. The script management and recommendation module includes a script knowledge base and a generative language model, which is connected to the multidimensional decision engine. It is triggered when the multidimensional communication risk level exceeds a first preset threshold, and performs operations such as matching scripts from the script knowledge base or calling the generative language model to generate scripts according to preset rules, the multidimensional communication risk level and the conversation context. The closed-loop optimization module, connected to the sentiment analysis engine and the script management and recommendation module, is used to monitor changes in user sentiment after a script is pushed as feedback data, and based on the feedback data, update the effectiveness score of the script knowledge base in the script management and recommendation module and / or fine-tune the generative language model.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images