Emotion-based voice interaction psychological support dialogue system, method, electronic device, storage medium

Abstract

The application discloses a psychological support dialogue system and method based on emotional voice interaction, an electronic device and a storage medium. The system comprises an emotional voice interaction module for performing intention and emotion perception on voice input; a customized variable module for obtaining a runtime state of the intention and emotion perception; a psychological support decision module for generating an output strategy according to safe priority judgment of the runtime state; a support action execution module for executing corresponding support actions according to the output strategy to update emotional reply voice and / or reply text; and a state updating module for updating the runtime state of the current round dialogue as historical information according to the updated reply text, the emotional reply voice and user feedback. The application can dynamically select appropriate psychological support dialogue reply functions.

Description

Technical Field

[0001] This application belongs to the technical field of intelligent mental health support systems, specifically relating to a psychological support dialogue system, method, electronic device, and storage medium based on emotional voice interaction. Background Technology

[0002] Speech emotion recognition (SER) is a technique for extracting emotion categories, emotion features, or emotion detection results from user speech. Emotional speech synthesis is a technique for generating synthetic speech with emotional expression based on emotion or style information. These two technologies can be combined into an emotional speech interaction module, possessing the capabilities of speech emotion perception (recognizing the emotional state in user speech), emotional speech generation (outputting emotionally synthesized speech that matches the context), and support for emotional multi-turn speech dialogue. Existing emotional speech interaction modules can be not only the classic combination of Whisper with SER models and emotional TTS, but also commercial and / or open-source engines such as FunAudio-Chat, EmotiVoice, Hume AI EVI, and ElevenLabs ConversationalAI. Existing emotional speech interaction modules typically improve model performance through feature representation, cross-modal fusion, contextual modeling, and training data expansion. For example, related solutions combine acoustic features, spectrograms, text semantics, dialogue context, and self-supervised pre-training or data augmentation methods to improve emotion classification accuracy and enhance model stability in noisy, cross-corpus, or imbalanced scenarios. There are relatively mature understanding failure handling mechanisms in general emotion-based voice interaction modules. For example, dialogue management methods based on confidence assessment, dialogue state tracking, and error recovery can guide the system to confirm, repeat the request, or roll back the process when there is noise interference, unstable recognition results, or semantic understanding failure.

[0003] In existing technologies, emotion-based voice interaction modules for mental health support dialogues have begun to utilize dialogue context, multimodal input, and language generation capabilities to provide supportive responses. For example, they can generate empathetic or guiding responses based on user expressions, conduct mental health assessments based on voice and multimodal features, or detect emotions from real-time voice and output notifications. However, mental health support dialogues differ from general chat; user states may change rapidly during the conversation, and the type, timing, and intensity of supportive information need to be adjusted accordingly. When a user's state cannot be reliably determined, it is not advisable to directly generate highly certain mental health support conclusions. If continuous risks or high-risk clues appear, it is also necessary to switch to risk alerts or human intervention prompts in a timely manner.

[0004] Existing technologies have several shortcomings. First, existing emotion-based voice interaction modules primarily focus on emotion recognition capabilities, unified speech and language modeling, or response generation quality, lacking a support decision-making mechanism that addresses mental health support dialogues. Second, while existing emotion-based voice interaction modules handle noise, comprehension failures, and ambiguities through confidence assessment, dialogue state tracking, and error recovery mechanisms, their crisis alerts or human referral mechanisms focus on speech comprehension repair, task-oriented dialogue management, emotion recognition performance improvement, or response control for identified crisis content, lacking a support action selection mechanism for each round of interaction in mental health support dialogues where safety is prioritized. Third, existing emotion-based voice interaction modules do not incorporate emotionally charged voice input, dialogue context, risk status, cue burden, and emotional voice output into the same round of decision-making during runtime, making it difficult to adjust the support type and intensity according to changes in each round of dialogue. Fourth, when persistent negative expressions, significant risk cues, or high-risk states occur, existing emotion-based voice interaction modules lack clear switching rules between the emotional response process and the risk alert / human intervention prompt process. Fifth, existing emotion-based voice interaction modules tend to repeatedly output cuees of the same intensity without considering the user's response to the cuees, potentially causing interaction redundancy or cue burden. Summary of the Invention

[0005] To address one or more of the aforementioned problems, this application provides a psychological support dialogue system based on emotional voice interaction in its first aspect, a psychological support dialogue method based on emotional voice interaction in its second aspect, an electronic device in its third aspect, and a storage medium in its fourth aspect, for realizing the function of dynamically selecting appropriate dialogue responses and support actions.

[0006] The technical solution of this application is as follows.

[0007] In the first aspect, a psychological support dialogue system based on emotion-based voice interaction includes: The emotion-based voice interaction module is used to perceive the intent and emotion of voice input. A custom variable module is used to obtain the runtime state of the aforementioned intention and emotion perception. The psychological support decision-making module is used to generate an output strategy by making a safety priority judgment based on the runtime state. A support action execution module is used to execute corresponding support actions according to the output strategy to update the emotional response voice and / or response text; The status update module is used to update the runtime status of the current round of dialogue as historical information based on the updated reply text, the emotional reply voice, and user feedback.

[0008] As one implementation of the first aspect, the emotion-based voice interaction module includes: The speech-to-text unit is used to convert speech input into transcribed text; The text semantic understanding unit is used to identify and extract semantic features from transcribed text; An acoustic feature extraction unit is used to extract acoustic features related to emotion expression from speech input; The emotion perception unit is used to output the emotional state corresponding to the current dialogue based on acoustic features, semantic features, and historical dialogue states. The response text generation unit is used to generate response text based on semantic features; The response speech synthesis unit is used to generate emotional response speech based on emotional state and response text.

[0009] Preferably, the speech-to-text unit, text semantic understanding unit, acoustic feature extraction unit, emotion perception unit, response text generation unit, and response speech synthesis unit are constructed as an emotion-speech interaction module in a multi-agent orchestration mechanism.

[0010] As one implementation of the first aspect, the runtime state is composed of custom variables; Custom variables include: Emotional state variables are used to represent a user's current emotional state, which includes emotion category, emotion intensity, and emotion confidence. Semantic risk variables are used to represent the strength of risk cues for semantic features in transcribed text; risk cues include semantic risk and historical risk. Historical risk variables are used to represent changes in the persistence, enhancement, or mitigation of risk cues during multiple rounds of dialogue; The dialogue stage variable is used to indicate the current stage of the dialogue; the dialogue stages include general communication, emotional expression, clarification and confirmation, support and guidance, and risk warning. Modal reliability variables are used to represent the speech quality of the speech input, the completeness of the speech transcription, and the completeness of the available modalities; a modality includes semantic features and acoustic features; Determine whether the uncertainty variable is stable; The prompt burden variable is used to represent the number of prompts that have been output, the number of times the same output strategy has been used consecutively, or the user's response to the prompt.

[0011] Preferably, the psychological support decision-making module includes: The decision initiation point management unit is used to set the upper limit of the preset processing time after obtaining voice input as the decision initiation point of the current round of dialogue; The runtime state receiving unit is used to receive the runtime state obtained by the custom variable module; The safety priority judgment unit is used to prioritize risk warnings or manual intervention prompts when high semantic risks and / or historical risks continue to rise during runtime. The uncertainty state processing unit is used to determine the uncertainty in the runtime state as unstable, and when there is no high semantic risk and / or the historical risk continues to rise, the priority decision is to adopt support actions such as clarification and confirmation, restatement or conservative response. The action matching unit is used to determine the uncertainty in the runtime state as stable, and when there is no high semantic risk and / or the historical risk continues to rise, it matches the support actions of empathetic response, CBT heuristic support, brief confirmation, and manual prompting based on semantic risk, historical risk, and dialogue stage. The prompt burden adjustment unit is used to adjust the support action to a short confirmation based on the corresponding number of prompts, the frequency of user response feedback, and the intensity of emotion when the historical risk in the runtime state has not increased. The output strategy generation unit is used to pass the supporting actions to the supporting action execution module and determine the corresponding content constraints, emotion intensity, and speech synthesis mode.

[0012] Furthermore, supporting actions include at least: Empathic responses are used to follow up on, paraphrase, or respond to the emotions expressed by users. Clarification and confirmation are used to confirm incomplete, ambiguous, or conflicting information. A conservative response is a low-intensity, non-judgmental response given by users when information is insufficient. CBT heuristic support is used to organize psychological support dialogue content in ways such as emotion naming, cognitive reappraisal, evidence testing, or behavioral activation. Risk warnings are used to provide clear and consistent safety tips or assistance resource suggestions. The manual intervention prompt is used to guide users to the human support path.

[0013] Furthermore, the content of user responses includes whether the user continues to respond, whether clarification and confirmation are completed, whether the prompts are accepted or ignored, whether the risk clues have changed, and changes in the emotional state of the current conversation.

[0014] The second aspect is a psychological support dialogue method based on emotional voice interaction, including: Get voice input; Intent and emotion perception of voice input; Obtain the runtime state for intention and emotion perception; Based on runtime state, a safety priority judgment is made to generate an output strategy; Execute corresponding support actions based on the output strategy to update the emotional response voice and / or response text; The runtime state of the current round of dialogue is updated as historical information based on the updated reply text, emotional reply voice, and user feedback.

[0015] In a third aspect, an electronic device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the psychological support dialogue method based on emotional voice interaction as described in the second aspect.

[0016] Fourthly, a storage medium, being a computer-readable storage medium, stores computer instructions for enabling a computer to implement the psychological support dialogue method based on emotional voice interaction as described in the second aspect.

[0017] Compared with the prior art, the advantages of this application are as follows: In psychological dialogue voice interaction, a safety-first runtime decision-making approach is adopted. A decision initiation point is set, and the runtime state of the current round of dialogue is described by customized variables such as emotional state, semantic risk, historical risk, dialogue stage, input reliability, judgment uncertainty, and cue burden. In each round of decision-making, the appropriate current supporting action is selected from the supporting actions according to the decision rules to adjust the output of this round. Then, the historical state is updated according to the user's subsequent response, risk changes, cue burden changes, and dialogue state changes for use in the next round of decision-making. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the structural framework of one of the psychological support dialogue systems based on emotion-based voice interaction in this application.

[0019] Figure 2 This is a flowchart illustrating one of the psychological support dialogue methods based on emotional voice interaction in this application.

[0020] Figure 3 This is a schematic diagram of the frame of one of the electronic devices in this application. Detailed Implementation

[0021] Referring to the illustrations, the principles of this application are illustrated by way of example implementation in a suitable operating environment. The following description is based on the illustrative specific embodiments of this application and should not be construed as limiting other specific embodiments not detailed herein, such as corresponding adjustments to the order of steps of the method of this application based on the principles of the technical concept.

[0022] like Figure 1 As shown in the figure, this application discloses a psychological support dialogue system based on emotional voice interaction, including a voice acquisition module, an emotional voice interaction module, a customized variable module, a psychological support decision-making module, a support action execution module, and a state update module.

[0023] The voice acquisition module is used to acquire voice input.

[0024] In this embodiment, the voice acquisition module obtains the user's voice signal from the microphone and processes it to obtain voice input in the form of digital audio data. Optionally, in this embodiment, the digital signal processing can incorporate a voice quality enhancement algorithm to further refine the accurate voice input.

[0025] The emotion-based voice interaction module is used to perceive the intent and emotion of voice input.

[0026] The emotion-based voice interaction module includes a speech-to-text unit, a text semantic understanding unit, an acoustic feature extraction unit, an emotion perception unit, a response text generation unit, and a response speech synthesis unit.

[0027] The speech-to-text unit is used to convert speech input into transcribed text using automatic speech recognition (ASR) technology. Preferably, in this embodiment, the speech input can be converted to text using methods including CTC, LAS, and RNN-T.

[0028] The text semantic understanding unit is used to identify and extract semantic features such as user intent, emotional semantics, and risk cues from transcribed text through natural language understanding technology (NLU), generative language models, and large language models.

[0029] The acoustic feature extraction unit is used to extract one or more acoustic features related to emotion expression from the speech input, such as sound intensity, fundamental frequency, speech rate, pause duration, prosody, and energy changes. Preferably, in this embodiment, the structure used for extracting acoustic features may include HMM-GMM, DNN, CNN, RNN, Transformer, n-gram grammar models, and hybrid structure models (such as GPT and BERT). Further optionally, before performing acoustic feature extraction, the speech input may undergo preprocessing such as pre-emphasis, frame-by-frame windowing, FFT, Mel filtering, logarithmic compression, and DCT.

[0030] The emotion perception unit is used to output the emotion category, emotion intensity, emotion confidence, and other emotion states corresponding to the speech input of the current dialogue based on one or more of acoustic features, semantic features, and historical dialogue states.

[0031] The response text generation unit is used to generate response text based on semantic features. Optionally, this embodiment may employ rule templates, retrieval-based generation, generative language models, or large language models to generate response text based on available action categories and content constraints. It may also further generate dialogue prompt text that is not directly used for output.

[0032] The response speech synthesis unit is used to generate emotional response speech based on emotional state and response text through technologies such as text-to-speech (TTS) and emotional speech synthesis.

[0033] Optionally in this embodiment, the speech-to-text unit, text semantic understanding unit, acoustic feature extraction unit, emotion perception unit, response text generation unit, and response speech synthesis unit can be constructed into an emotion speech interaction module using a multi-agent orchestration mechanism. This organizes processes such as semantic understanding, response generation, tool invocation, and output checking. The multi-agent orchestration is not limited to specific or general self-supervised speech representations, large language models, agent frameworks, speech recognition models, speech synthesis models, etc.

[0034] The runtime state of the emotion-based voice interaction module includes, but is not limited to, emotional state, semantic risk, historical risk, dialogue stage, modal reliability, judgment uncertainty, and prompting burden.

[0035] A custom variable module is used to acquire custom variables corresponding to the intention and emotion perception of the emotion-based voice interaction module, serving as the runtime state of the module. In this embodiment, one or more of the following units—speech-to-text unit, text semantic understanding unit, acoustic feature extraction unit, emotion perception unit, response text generation unit, and response speech synthesis unit—each connect their respective inputs and / or outputs to the custom variable module. The custom variable module processes the corresponding outputs to obtain the corresponding custom variables. In this embodiment, the runtime state of the current round includes historical information of partial runtime states formed from previous rounds of dialogue. Custom variables can be represented using continuous scores, discrete levels, or structured labels. Custom variables can be normalized to values ​​within a preset range or converted into low, medium, and high discrete levels. In other embodiments, custom variables can be further determined based on rule bases, statistical models, or manually configured parameters.

[0036] In this embodiment, custom variables may include, but are not limited to: (1) Emotional state variables are used to represent the user's current emotional state, such as emotional category, emotional intensity, and emotional confidence. The judgment of the background of the user's current emotional expression affects the decision on the tone of support, the intensity of support, and whether further guidance is needed. For example, the emotional category may include depressed, anxious, and stable. The emotional intensity and emotional confidence can be obtained by normalizing the scores. The emotional state variables are obtained based on the output of the emotional perception unit to obtain the emotional category, emotional intensity, and emotional confidence. (2) Semantic risk variables are used to represent the intensity of risk cues in the transcribed text. They can be represented by normalized scores or discrete levels of low, medium and high. They are the basis for capturing risk cues explicitly expressed by users and for triggering risk warnings or manual intervention prompts. Semantic risk variables are obtained based on the risk cue output of the text semantic understanding unit. (3) Historical risk variables are used to represent the changes in the continuity, enhancement or mitigation of risk cues in multi-turn dialogues (a kind of partial runtime state formed by multiple rounds of dialogue in the past). They can be represented by trend labels or cumulative scores. They are used to identify trends that cannot be reflected in a single round of input, so as to avoid making short-sighted judgments based on the current sentence alone. Historical risk variables are obtained by statistical analysis of the output records of risk cues in multiple rounds of text semantic understanding units. (4) Dialogue stage variables are used to represent the current stage of the dialogue. In this embodiment, the dialogue stages include general communication, emotion expression, clarification and confirmation, support and guidance, and risk warning. They are represented by discrete state labels to identify the timing of constraints so as to facilitate subsequent support action decisions (a kind of partial runtime state formed by multiple rounds of dialogue). For example, in the clarification and confirmation stage, semantic confirmation is prioritized instead of directly outputting strong guidance content. The dialogue stage variables are obtained by identifying the dialogue features and determining the current dialogue stage based on the output of one or more units in the text semantic understanding unit, emotion perception unit, response text generation unit, and response speech synthesis unit. (5) Modal reliability variables are used to represent the speech quality of the speech input, the completeness of the speech transcription, and the completeness of the available modalities. They are represented by scores or levels and can handle noise, incomplete transcription, and missing clues. When the modal reliability is insufficient and there is no clear high-risk semantics, a clarification confirmation or conservative response stage is triggered. The modal reliability variables are obtained by sampling and measuring the input of the speech to text unit to obtain the speech quality of the speech input. The modal reliability variables are obtained by key information backtracking and other methods based on the transcribed text output by the speech to text unit and its speech input. The modal reliability variables are obtained by performing modal dimension statistical analysis on the semantic feature output of the text semantic understanding unit and the acoustic feature output of the acoustic feature extraction unit to obtain the completeness of the available modalities. (6) Determine the uncertainty variable, which is used to indicate whether the current emotion perception judgment and / or risk judgment is stable and reliable. It is represented by uncertainty score or level, which can handle the situation where there is conflict between speech and text, insufficient confidence, and large fluctuations in judgment results, and prevent over-reliance on unstable judgments. The uncertainty variable is obtained by performing consistency judgment and confidence calculation based on the output of the text semantic understanding unit and the emotion perception unit. (7) Prompt burden variable, which is used to represent the number of prompts that have been output within the preset time window, the number of times the same output strategy has been used consecutively, or the user's response to the prompts (a kind of runtime state formed by multiple rounds of dialogue in the past). It can adjust the frequency and intensity of prompts and reduce the interaction burden caused by repeated prompts, but it is not used to cancel necessary risk prompts in high-risk scenarios. The prompt burden variable is obtained by statistical analysis of the prompt content output by the reply text generation unit and the user's response to the prompts understood by the text semantic understanding unit.

[0037] Optionally in this embodiment, the runtime state will be organized into customized variables and output to the psychological support decision-making module. Modal reliability variables and judgment uncertainty variables are used to support the downgrading or clarification confirmation mechanism of support actions in real interactive environments. The cue burden variable is used to avoid excessively repeating the same type of cue when no state change is observed. The scope of the cue burden variable is to adjust the frequency and intensity of support in non-high-risk scenarios. When the semantic risk variable or historical risk variable reaches the set high-risk condition, risk cueing or manual intervention cueing will still be executed first.

[0038] The psychological support decision-making module is used to generate output strategies by performing safety priority judgments and other processing based on the runtime state.

[0039] In this embodiment, the psychological support decision-making module does not require the pre-trained policy network to output the highest-probability follow-up action as a necessary step, nor does it take proactive follow-up questions that guide the user to express further as its sole objective. Unlike general emotion-based speech dialogue models, the psychological support decision-making module processes runtime states, support action selection, and output adjustment decisions, rather than training or updating the parameters of the basic recognition model, response generation model, or speech synthesis model. Furthermore, the psychological support decision-making module does not require the large language model to output a single risk score before proceeding to rejection, clarification, or normal response. In psychological speech dialogue scenarios, this application comprehensively considers runtime states such as semantic risk, historical risk, modal reliability, judgment uncertainty, and cueing burden to select the psychological support action, content constraints, emotion intensity, and speech synthesis mode for the current round of dialogue.

[0040] The psychological support decision-making module includes a decision initiation point management unit, a runtime status receiving unit, a safety priority judgment unit, an uncertain state handling unit, a support action matching unit, a prompt burden adjustment unit, and an output strategy generation unit.

[0041] The decision initiation point management unit is used to set the upper limit of the preset processing time after obtaining voice input as the decision initiation point for the current round of dialogue. The decision-making process for output strategy only begins when the decision initiation point is reached. In this embodiment, during psychological dialogue voice interaction, users typically present their current state gradually through multiple rounds of voice expression. The emotional intensity, expression willingness, risk cues, and acceptable support methods of the same user may change in adjacent rounds. Therefore, the processing window expiration time after each round of voice input is set as the decision initiation point, rather than making a one-time judgment only at the beginning of the dialogue or at a fixed time interval. This allows the current input and historical runtime state information to be reread in each round of dialogue.

[0042] The runtime state receiving unit is used to receive the runtime state obtained by the custom variable module.

[0043] The safety priority judgment unit is used to prioritize risk warnings or manual intervention prompts when high semantic risks and / or historical risks continue to rise during runtime.

[0044] In other optional embodiments, the safety priority judgment unit may also use a comprehensive risk based on the weighted sum of emotional state, semantic risk, and historical risk; wherein emotional state, semantic risk, and historical risk are all represented by normalized values, and the weights can be set according to preset rules or obtained through adaptive learning.

[0045] The uncertainty state processing unit is used to determine the uncertainty in the runtime state as unstable, and when there is no high semantic risk and / or the historical risk continues to rise, it prioritizes the adoption of supporting actions such as text generation and / or speech synthesis strategies such as clarification and confirmation, restatement or conservative response.

[0046] The support action matching unit is used to determine the uncertainty in the runtime state as stable, and when there is no high semantic risk and / or a continuous increase in historical risk, it matches support actions such as empathic response, CBT heuristic support, brief confirmation, and manual prompts based on semantic risk, historical risk, and dialogue stage.

[0047] The burden adjustment unit is used to adjust support actions such as brief confirmations based on the number of prompts, the number of consecutive identical strategies, or the frequency and intensity of user response feedback when the historical risk in the runtime state has not yet increased.

[0048] The output strategy generation unit is used to pass the supporting actions to the supporting action execution module and determine the corresponding content constraints, emotion intensity, and speech synthesis mode.

[0049] In this embodiment, the execution meaning, setting rationale, and applicable states of various supporting actions are as follows: (a) Empathic response: responding to, restating or emotionally responding to the user’s expression, serving as a basic support action to maintain natural dialogue and a sense of being understood. It is used in low-risk situations where the modality is reliable and the user is making general or mild emotional expressions. (b) Clarification and confirmation: to confirm incomplete, ambiguous or conflicting information, thereby improving the understanding of the current round of dialogue and avoiding the direct output of strongly guiding content based on unreliable modalities. It is used when there is a lot of noise in the voice input, the transcribed text is incomplete, the risk clues are conflicting or the judgment is uncertain. (c) Conservative response: When information is insufficient, a low-intensity, non-judgmental response is given. This serves to maintain the continuity of the dialogue and reduce the risk of inappropriate response content when the running status cannot be fully judged. It is used when the current running status is insufficient to support strong guidance but still needs to continue to accept user expressions. (d) CBT heuristic support organizes psychological support dialogue content through methods such as emotion naming, cognitive reappraisal, evidence testing or behavioral activation. It serves to provide structured guidance for moderate support needs without outputting professional conclusions. It is used for medium-level support needs, persistent negative expressions, and situations where users are willing to continue communicating. (e) Risk warning: Output clear and stable safety warnings or help resource warnings, which serve to switch ordinary support actions to safety boundary warnings when the risk increases, and are used to clarify the risk semantics, the semantic risk is high, or the historical risk continues to rise. (f) Manual intervention prompts prompt users to switch to the manual support path, which serves to guide the interaction to the manual support channel when psychological support decision-making is insufficient, and is used to deal with high-risk conditions or situations where psychological support decision-making support capabilities are insufficient.

[0050] In this embodiment, a specific operation process of the psychological support decision-making module is as follows: the decision initiation point management unit obtains the decision initiation point of the current round, and then starts the decision-making process to be executed sequentially; first, the runtime state receiving unit receives the runtime state obtained by the current round customized variable module; then, the safety priority judgment unit judges from the runtime state whether there is a clear high semantic risk and / or a continuous increase in historical risk. If there is a clear high semantic risk and / or a continuous increase in historical risk, it prioritizes the decision of whether to issue a risk warning or manual intervention warning; if there is no clear high semantic risk and / or no continuous increase in historical risk, it enters the uncertainty state processing unit; if the uncertainty state processing unit judges from the runtime state that the uncertainty is unstable and there is no high semantic risk and / or a continuous increase in historical risk, it prioritizes... The system selects an appropriate response strategy from among clarification and confirmation, restatement, or conservative responses; otherwise, it proceeds to the support action matching unit. Then, if the uncertainty is stable and there is no high semantic risk or continuously increasing historical risk during runtime, the support action matching unit selects support actions such as empathic responses, CBT heuristic support, or brief confirmation based on semantic risk, historical risk, and dialogue stage. Next, if the historical risk has not increased during runtime, the prompt burden adjustment unit adjusts the frequency and emotional intensity of prompts based on the corresponding number of prompts, the number of consecutive identical strategies, or user response feedback, or adjusts the output to a brief confirmation decision. Finally, the output strategy generation unit outputs the support actions, content constraints, emotional intensity, and speech synthesis mode for this round, providing the support action execution module with the necessary updates to the corresponding response text and emotional response speech.

[0051] The above process concretizes the decision initiation point, customized variables, set of supporting actions, decision-making methods, and feedback relationships in real-time intervention into the supporting action selection process during the safety-first runtime of psycho-voice dialogue. This feedback updates the dialogue output content and method from the runtime state and participates in subsequent decisions as part of the runtime state in the next round. Therefore, when the user state, the quality of voice input, risk cues, cue burden, etc., change, what changes is the type, content, intensity, or expression of the supporting actions in the current round, rather than modifying the underlying model parameters.

[0052] In this embodiment, optionally, the decision-making process can use a set threshold and / or a regular expression to describe the logical relationship of the decision-making mechanism. The decision-making strategy during implementation can be achieved by a combination of fixed rules, configurable rule engines, classification models, decision trees, probability models, or rule-generating language models, and all of them retain the safety priority constraint, that is, when the high semantic risk or historical risk continues to rise, the risk warning or manual intervention warning judgment is given priority, and the warning burden shall not cancel the necessary high-risk warning.

[0053] A support action execution module is used to execute corresponding support actions according to the output strategy to update the emotional response speech and / or response text. In this embodiment, the inputs to the support action execution module include, but are not limited to, action category, content constraints, emotion intensity, and speech synthesis mode; the outputs include updated response text, prompt text, and emotional response speech. The support action execution module completes the execution of the dialogue response in this round according to the action category, content constraints, emotion intensity, and speech synthesis mode determined by the psychological support decision module.

[0054] Optionally in the embodiment, if the category of the support action output by the psychological support decision module is clarification and confirmation, the content constraint is to avoid directly judging the user's state and prioritize confirming the meaning expressed by the user, the emotional intensity is low, and the speech synthesis mode is clear confirmation mode, then the support action execution module can generate a reply text for confirming the meaning, and the emotional reply speech is in the form of slow speech speed, clear pronunciation, and clear pauses.

[0055] In this embodiment, the support actions can optionally be categorized into six types: empathic response, clarification confirmation, conservative response, CBT heuristic support, risk warning, and manual intervention warning. Brief confirmation is a low-intensity execution form when the warning burden is high. The basis for setting the support actions is that: psycho-voice dialogue needs to handle both general emotional expressions and unreliable states in real voice input; it needs to provide structured guidance under moderate support needs and clarify system boundaries when risks increase. Empathic response is a common support method in emotional dialogue, empathetic dialogue, and psychological comfort dialogue. Clarification confirmation originates from the understanding and repair approach in voice dialogue, and this embodiment uses it for low reliability or high uncertainty scenarios. Conservative response is used in scenarios where information is insufficient but dialogue continuity still needs to be maintained to reduce the risk of inappropriate support caused by high-certainty output. CBT heuristic support references the ideas of structured psychological support and immediate adaptive support actions, and is only used as a content organization strategy. Risk warning and manual intervention warning are used for decision switching when high semantic risk and historical risk increase, or when automatic support capabilities are insufficient. CBT heuristic support actions only indicate content organization strategies used for interactive support, and do not imply making professional judgments or replacing human support; risk warnings and human intervention prompts indicate output control methods in high-risk scenarios, and do not imply making professional conclusions.

[0056] The state update module is used to save and update the current round of dialogue's runtime state based on the updated response text and emotional response voice output by the supporting action execution module, as well as the current round's runtime state, the selected supporting action, and subsequent user feedback. This updated runtime state is then fed back to the next round of dialogue to build a runtime state for use as historical information.

[0057] In this embodiment, user follow-up feedback refers to the user response content observable in the next round of dialogue or a preset time window after the current action-supporting updated reply text and emotional reply voice output. This may include whether the user continues to respond, whether clarification and confirmation are completed, whether the prompt is accepted or ignored, whether risk cues change, and observable changes in emotional state during this round of dialogue. User follow-up feedback is not an evaluation of the emotional reply dialogue, but rather a state feedback used for decision-making in the next round of reply dialogue.

[0058] In this embodiment, the state update module can update the historical information required for subsequent rounds of decision-making based on user feedback. For example, after the user completes clarification and confirmation, the uncertainty of the next round of judgment can be reduced; for example, after the user continuously ignores or rejects similar prompts, the prompting burden can be increased; for example, after semantic risk increases, historical risk can be increased; for example, after the dialogue enters the support guidance or risk prompt stage, the dialogue stage can be updated accordingly, etc. The action execution support module and the psychological support decision-making module are the core modules for achieving the invention's objectives.

[0059] The overall concept of this application is to add a decision-making mechanism for psychological dialogue scenarios on top of the existing emotion-based voice interaction model, without altering its output text, emotion perception results, and related intermediate states. This decision-making mechanism determines in each round of dialogue whether the runtime state is suitable for direct support, whether clarification is needed first, whether the intensity of automatic judgment needs to be reduced, and whether a risk warning or manual intervention process should be initiated. Based on these decisions, more reliable response text and emotional response voice are obtained through updates. This application prioritizes safety, continuity, and support adaptability in dialogue generation. Because psychological dialogue requires maintaining stable responses throughout a relatively long conversation, it avoids outputting inappropriate content when there is low-quality input or increased risk. User responses are selected based on observable results within the current round or a short time window, and these responses are recorded and fed back into the runtime state of the next round, thus forming a closed loop for psychological support decisions. This application is neither a solution that selects the highest-probability follow-up action based on emotional context, nor a solution that enters a rejection, clarification, or normal response process based on a single risk score. Instead, it is a solution that achieves psychological dialogue support output centered on runtime state, safety priority rules, support actions, prompting burden, user feedback response, and voice synthesis control.

[0060] Compared with the prior art, the advantages of this application are as follows: In psychological dialogue voice interaction, a safety-first runtime decision-making approach is adopted. A decision initiation point is set, and the runtime state of the current round of dialogue is described by customized variables such as emotional state, semantic risk, historical risk, dialogue stage, input reliability, judgment uncertainty, and cue burden. In each round of decision-making, the appropriate current supporting action is selected from the supporting actions according to the decision rules to adjust the output of this round. Then, the historical state is updated according to the user's subsequent response, risk changes, cue burden changes, and dialogue state changes for use in the next round of decision-making. This approach builds upon basic emotional voice interaction capabilities to provide psychological support decisions. Compared to existing technologies, it focuses on the utilization of runtime states without requiring changes to the speech recognition, emotion recognition, language generation, or emotional speech synthesis models themselves. It can adapt to different types of basic emotional voice interaction models or agents. It selects the current support action by combining emotional state, risk cues, historical states, input reliability, and judgment uncertainty. When automatic judgment is insufficient, it can perform clarification confirmation, conservative response, or prompt downgrade. When a clear semantic risk occurs, it can prioritize risk prompts or manual intervention, thereby avoiding reliance on a single emotion category output. The hierarchical support action settings can select support actions based on different runtime states, thereby reducing the possibility of unreliable judgments triggering inappropriate support outputs. By incorporating historical risk changes, cue burden, and user feedback into the runtime state update process, policy selection is avoided based solely on single-turn input, thereby improving support adaptability in continuous dialogue. This reduces repetitive or overly dense cueing when historical risk is not high, while maintaining necessary safety-first response decisions when historical risk is high. Mapping support actions to corresponding speech synthesis modes generates emotional response speech outputs for those modes, ensuring that the speech expression remains consistent with the support actions performed without altering the structure or parameters of the speech synthesis model itself.

[0061] like Figure 2 As shown in the figure, this application discloses a psychological support dialogue method based on emotional voice interaction, which includes the following steps.

[0062] S1. Obtain voice input.

[0063] S2. Perform intent and emotion perception on voice input. Specifically, this includes: S21. Convert voice input into transcribed text using automatic speech recognition technology; S22. Using natural language understanding technology, generative language models, large language models, etc., identify and extract semantic features such as user intent, emotional semantics, and risk cues from transcribed text; S23. Extract one or more acoustic features related to emotion expression from speech input, such as intensity, fundamental frequency, speech rate, pause duration, prosody, and energy changes. S24. Based on one or more of acoustic features, semantic features, and historical dialogue states, output the emotional state, such as the emotion category, emotion intensity, and emotion confidence level of the speech input corresponding to the current dialogue. S25. Generate response text based on semantic features; S26. Using technologies such as text-to-speech synthesis and emotional speech synthesis, generate emotional response speech based on emotional state and response text.

[0064] S3. Obtain the corresponding custom variables as runtime states when performing intention and emotion perception.

[0065] S4. Generate an output strategy by performing safety priority judgments and other processing based on the runtime state. Specifically, this includes: S41. Set the upper limit of the preset processing time after obtaining voice input as the decision start point of the current round of dialogue. Only when the decision start point is reached will the decision process of output strategy begin. S42, Receive runtime status; S43. When high semantic risks and / or historical risks continue to rise during runtime, priority should be given to issuing risk warnings or manual intervention. S44. When the uncertainty is determined to be unstable during runtime, and there is no high semantic risk and / or a continuous increase in historical risk, the priority decision is to adopt a text generation and / or speech synthesis strategy of clarification and confirmation, restatement or conservative response. S45. When the historical risk in the runtime state has not yet increased, adjust the support actions such as brief confirmation according to the corresponding number of prompts, the number of consecutive times of the same strategy, or the frequency and emotional intensity of the adjustment prompts in the user's response feedback. S46. Pass the supporting action to the next step and determine the corresponding content constraints, emotion intensity, and speech synthesis mode.

[0066] S5. Execute corresponding support actions according to the output strategy to update the emotional response voice and / or response text.

[0067] S6. Based on the updated reply text and emotional reply voice, as well as the runtime state of the current round, the selected support action, and subsequent user feedback, save and update it to the runtime state of the current round of dialogue, and then feed it back to the next round of dialogue to build the runtime state for use as historical information.

[0068] The methods described above are executed based on the corresponding emotional voice interaction-based psychological support dialogue systems in the foregoing embodiments, and have the beneficial effects of the corresponding system embodiments, which will not be repeated here.

[0069] like Figure 3As shown, based on the same inventive concept, corresponding to any of the above embodiments, this application also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the above-mentioned psychological support dialogue method based on emotional voice interaction.

[0070] Specifically, the device includes: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected within the device via the bus 1050.

[0071] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), GPU (Graphics Processing Unit), or one or more integrated circuits, to implement relevant programs and achieve the technical solutions provided in the embodiments of this specification.

[0072] The memory 1020 can be implemented in the form of ROM (Read-Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and called by the processor 1010. The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices can include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices can include displays, projectors, speakers, vibrators, indicator lights, etc.

[0073] The communication interface 1040 is used to connect the communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB (Universal Serial Bus), network cable, etc.) or wireless means (such as mobile network, WIFI (Wireless Fidelity), Bluetooth, etc.).

[0074] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0075] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0076] The electronic devices described above are used to implement the corresponding psychological support dialogue methods based on emotional voice interaction in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0077] Based on the same inventive concept, corresponding to any of the above embodiments, this application also discloses a computer-readable storage medium that stores computer instructions for enabling a computer to implement the above-described psychological support dialogue method based on emotional voice interaction.

[0078] The computer-readable storage medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium, which can be used to store information accessible by a computing device. The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to implement the psychological support dialogue method based on emotion-based voice interaction as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0079] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments or combinations thereof, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A psychological support dialogue system based on emotion-based voice interaction, characterized in that, include: The emotion-based voice interaction module is used to perceive the intent and emotion of voice input. A custom variable module is used to obtain the runtime state of the aforementioned intention and emotion perception. The psychological support decision-making module is used to generate an output strategy by making a safety priority judgment based on the runtime state. A support action execution module is used to execute corresponding support actions according to the output strategy to update the emotional response voice and / or response text; The status update module is used to update the runtime status of the current round of dialogue as historical information based on the updated reply text, the emotional reply voice, and user feedback.

2. The psychological support dialogue system based on emotional voice interaction according to claim 1, characterized in that, The emotion-based voice interaction module includes: The speech-to-text unit is used to convert speech input into transcribed text; A text semantic understanding unit is used to identify and extract semantic features from the transcribed text; An acoustic feature extraction unit is used to extract acoustic features related to emotion expression from speech input; An emotion perception unit is used to output the emotion state corresponding to the current dialogue based on the acoustic features, the semantic features, and the historical dialogue state. The response text generation unit is used to generate response text based on semantic features; The response speech synthesis unit is used to generate emotional response speech based on emotional state and response text.

3. The psychological support dialogue system based on emotional voice interaction according to claim 2, characterized in that, The speech-to-text unit, the text semantic understanding unit, the acoustic feature extraction unit, the emotion perception unit, the response text generation unit, and the response speech synthesis unit are constructed as the emotion speech interaction module in a multi-agent orchestration mechanism.

4. The psychological support dialogue system based on emotional voice interaction according to claim 1, characterized in that, Runtime state is composed of custom variables; The custom variables include: Emotional state variables are used to represent a user's current emotional state, which includes emotion category, emotion intensity, and emotion confidence. Semantic risk variables are used to represent the strength of risk cues for semantic features in transcribed text; the risk cues include semantic risk and historical risk. Historical risk variables are used to represent changes in the persistence, enhancement, or mitigation of risk cues during multiple rounds of dialogue; The dialogue stage variable is used to indicate the current stage of the dialogue; the dialogue stages include general communication, emotional expression, clarification and confirmation, support and guidance, and risk warning. Modal reliability variables are used to represent the speech quality of the speech input, the completeness of the speech transcription, and the completeness of the available modalities; the modalities include semantic features and acoustic features; Determine whether the uncertainty variable is stable; The prompt burden variable is used to represent the number of prompts that have been output, the number of times the same output strategy has been used consecutively, or the user's response to the prompt.

5. The psychological support dialogue system based on emotional voice interaction according to claim 4, characterized in that, The psychological support decision-making module includes: The decision initiation point management unit is used to set the upper limit of the preset processing time after obtaining voice input as the decision initiation point of the current round of dialogue; The runtime state receiving unit is used to receive the runtime state obtained by the custom variable module; The safety priority judgment unit is used to prioritize risk warnings or manual intervention prompts when high semantic risks and / or historical risks continue to rise during runtime. An uncertain state processing unit is used to determine in the runtime state that the uncertainty is unstable and there is no high semantic risk and / or the historical risk continues to rise, and to prioritize the adoption of the support action of clarification confirmation, restatement or conservative response. The action matching unit is used to determine the uncertainty in the runtime state as stable, and when there is no high semantic risk and / or the historical risk continues to rise, it matches the support actions of empathetic response, CBT heuristic support, brief confirmation, and manual prompting based on semantic risk, historical risk, and dialogue stage. The prompt burden adjustment unit is used to adjust the support action to a short confirmation based on the corresponding number of prompts, the frequency of user response feedback, and the intensity of emotion when the historical risk in the runtime state has not increased. The output strategy generation unit is used to pass the supporting actions to the supporting action execution module and determine the corresponding content constraints, emotion intensity, and speech synthesis mode.

6. The psychological support dialogue system based on emotional voice interaction according to claim 5, characterized in that, Supported actions include at least: Empathic responses are used to follow up on, paraphrase, or respond to the emotions expressed by users. Clarification and confirmation are used to confirm incomplete, ambiguous, or conflicting information. A conservative response is a low-intensity, non-judgmental response given by users when information is insufficient. CBT heuristic support is used to organize psychological support dialogue content in ways such as emotion naming, cognitive reappraisal, evidence testing, or behavioral activation. Risk warnings are used to provide clear and consistent safety tips or assistance resource suggestions. The manual intervention prompt is used to guide users to the human support path.

7. The psychological support dialogue system based on emotional voice interaction according to claim 5, characterized in that, User responses include whether the user continues to respond, whether clarification and confirmation are completed, whether the prompt is accepted or ignored, whether the risk clues have changed, and changes in the emotional state of the user during this round of conversation.

8. A psychological support dialogue method based on emotional voice interaction, characterized in that, include: Intent and emotion perception of voice input; Obtain the runtime state for performing the aforementioned intention and emotion perception; Based on the runtime state, a safety priority judgment is made to generate an output strategy; Execute corresponding support actions according to the output strategy to update the emotional response voice and / or response text; Based on the updated reply text, the emotional reply voice, and user feedback, update the runtime state of the current round of dialogue as historical information.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the psychological support dialogue method based on emotional voice interaction as described in claim 8.

10. A storage medium, which is a computer-readable storage medium, characterized in that, The computer contains computer instructions for enabling the computer to implement the psychological support dialogue method based on emotional voice interaction as described in claim 8.

Images