Voice interaction method, electronic device, and storage medium

Abstract

The application relates to the technical field of voice interaction, and discloses a voice interaction method, an electronic device and a storage medium. The method comprises the following steps: acquiring playing state information of a voice assistant, dialogue state information and collected streaming audio data; performing feature extraction according to the streaming audio data in combination with the playing state information and the dialogue state information to obtain multi-dimensional audio features; calculating an interaction direction score according to the multi-dimensional audio features; and rejecting the streaming audio data when the interaction direction score meets a preset rejection condition. The embodiment of the application can accurately identify the interaction intention of a user, and reduce the false triggering of non-interaction voice and the missed identification of effective instructions.

Description

Technical Field

[0001] This application relates to the field of voice interaction technology, and in particular to a voice interaction method, electronic device and storage medium. Background Technology

[0002] In open environments such as homes or offices, full-duplex voice assistant systems commonly employ speaker separation technology, automatic speech recognition confidence thresholding, or target speaker binding mechanisms to filter environmental interference. However, these methods have fundamental flaws: speaker separation technology can only identify the speaker's identity at the source of the speech, but cannot determine whether specific speech content is intended for interaction with the voice assistant. For example, when a user is having a casual conversation with family members and issuing commands to the voice assistant in the same scenario, the system struggles to distinguish which utterances are valid commands for the assistant. This can lead to the voice assistant mistakenly identifying casual conversation as commands, triggering unintended behavior, or misinterpreting explicit user commands as background noise and ignoring them, resulting in interrupted interaction. Furthermore, confidence thresholding methods rely solely on the reliability of the recognition results and cannot effectively handle high-confidence but non-assistant speech content; target speaker binding mechanisms require pre-fixed user identities, making them unsuitable for dynamic multi-user interaction scenarios. These limitations severely restrict the accuracy and user experience of existing technologies in real-world environments. Summary of the Invention

[0003] The purpose of this application is to provide a voice interaction method, electronic device, and storage medium that can accurately identify the user's interaction intent and reduce the false triggering of non-interactive voice and the omission of valid commands.

[0004] This application provides a voice interaction method, including: Acquire playback status information, dialogue status information, and collected streaming audio data from the voice assistant; Based on the streaming audio data, combined with the playback status information and the dialogue status information, feature extraction is performed to obtain multi-dimensional audio features; the multi-dimensional audio features include acoustic interference features, text confidence features, dialogue status features, and semantic orientation features; Based on the multidimensional audio features, an interaction orientation score is calculated; the interaction orientation score is used to characterize whether the streaming audio data has an interactive intent toward the voice assistant. When the interactive pointing score meets the preset rejection conditions, the streaming audio data is rejected.

[0005] In some embodiments, the feature extraction based on the streaming audio data, combined with the playback status information and the dialogue status information, includes: Based on the playback status information, acoustic feature recognition is performed on the streaming audio data to extract interfering audio from the streaming audio data and obtain the acoustic interference features. Confidence features and semantic features are extracted from the identified text obtained by recognizing the streaming audio data to obtain the text confidence features and the semantic orientation features. The dialogue state information is subjected to state feature extraction to obtain the dialogue state features.

[0006] In some embodiments, calculating the interaction-oriented score based on the multidimensional audio features includes: The acoustic interference features, the text confidence features, the dialogue state features, and the semantic orientation features are fused to obtain multi-dimensional fused features; The multidimensional fusion features are input into the scoring model to obtain the output interaction-oriented score.

[0007] In some embodiments, the voice interaction method further includes: When the interaction pointing score does not meet the rejection condition and the text confidence feature does not meet the preset confidence condition, the identified text obtained by recognizing the streaming audio data is subjected to error correction processing.

[0008] In some embodiments, the error correction processing of the identified text obtained by recognizing the streaming audio data includes: Extract word fragments from the identified text whose word-level confidence is lower than a preset word-level confidence threshold; Based on at least one of the following: the N-best candidates of the identified text, phonetic similar words, domain entity library, current dialogue context, and current task state, at least one error correction candidate is generated for the word fragment. Calculate the comprehensive score of the error correction candidates, and select the error correction candidate with the highest comprehensive score as the target error correction candidate; the comprehensive score is determined based on at least one of the following: the pronunciation similarity between the error correction candidate and the word segment, the matching degree between the error correction candidate and the current dialogue context, the prior probability of the error correction candidate in its domain, and the editing cost required to modify it into the error correction candidate; When the overall score is higher than a preset score threshold and the action risk level of the action corresponding to the target error correction candidate is lower than a preset risk threshold, the action corresponding to the target error correction candidate is executed. When the overall score is not higher than the score threshold, or the action risk level of the action corresponding to the target error correction candidate is not lower than the risk threshold, the expression in the candidate audio set is tested within a preset audio testing window; the candidate audio set includes at least the target error correction candidate and the corresponding confirmation expression.

[0009] In some embodiments, before generating the error correction candidates for the word fragments, the method further includes: Based on the semantic pointing features, determine whether the identified text contains an interruption intent; Upon detecting the interruption intention, based on the source direction stability and audio energy change characteristics of the streaming audio data at the time corresponding to the interruption intention, a temporary boundary between the foreground speech and the background speech is marked, and an acoustic feature template of the foreground speech is determined; the acoustic feature template is used to perform micro-segment filtering on the streaming audio data after the interruption intention.

[0010] In some embodiments, before generating the error correction candidates for the word fragments, the method further includes: Based on the current dialogue state of the voice assistant, determine the slot type of the slot to be filled. Based on the slot type, a whitelist of error correction candidates is established; the whitelist of error correction candidates contains words that match the slot type; The N-best candidates of the identified text are filtered using the error correction candidate whitelist, and the filtered N-best candidates are used as the error correction candidate source for the word segment.

[0011] In some embodiments, the voice interaction method further includes: Collect user feedback information on subsequent behaviors related to the rejection result; Based on the subsequent behavioral feedback information, false rejection events are identified to obtain the corresponding identification results; Based on the recognition result, update the judgment threshold in the rejection condition, or use the subsequent behavior feedback information as a training sample.

[0012] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described voice interaction method.

[0013] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described voice interaction method.

[0014] The beneficial effects of this application are as follows: Based on the playback status information and dialogue status information of the voice assistant, and the collected streaming audio data, feature extraction is performed to obtain multi-dimensional audio features. By comprehensively analyzing the multi-dimensional audio features, considering multiple dimensions such as acoustic environment, recognition reliability, dialogue context and semantic orientation, an interaction orientation score representing whether the streaming audio data has an interactive intention towards the voice assistant is calculated. When the interaction orientation score meets the preset rejection conditions, the streaming audio data is rejected. In complex scenarios where users may give instructions to the assistant or chat with others, it can accurately determine whether the voice is directed towards the voice assistant, reducing the false triggering of non-interactive voice and the missed recognition of valid instructions. Attached Figure Description

[0015] Figure 1 This is a flowchart of the voice interaction method provided in the embodiments of this application.

[0016] Figure 2 This is a flowchart of the specific method of step S102 provided in the embodiments of this application.

[0017] Figure 3 This is a flowchart of the specific method of step S103 provided in the embodiments of this application.

[0018] Figure 4 This is a flowchart of a method for error correction processing of recognized text obtained from recognized streaming audio data, provided in an embodiment of this application.

[0019] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and drawings are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application. Furthermore, the information, data, and signals involved in the embodiments of this application are all authorized by relevant parties or have been fully authorized by all parties, and the collection, use, and processing of related data comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0023] In open environments, a key technical challenge for voice assistant systems lies in their inability to accurately distinguish whether a user's voice is directed at them. Existing technologies rely on speaker separation, ASR confidence thresholds, or target speaker binding mechanisms, but these methods can only identify the main source of the voice and cannot analyze the interactive direction of the voice content. When the same user alternates between assistant commands and casual conversation, the system lacks the ability to judge the contextual relationship of semantic intent, leading to the mis-triggering of non-target voice or the omission of valid commands. This problem directly weakens the reliability and accuracy of voice interaction, causing inefficient use of system resources and interruption of user operation processes. For example, in a family living room scenario, a user issues a volume adjustment command to the voice assistant while discussing TV program content with family members. The system collects streaming audio data containing multi-person dialogue, where the user tells the assistant "turn down the volume" and immediately turns to ask a family member, "What do you think of the plot?" Speaker separation technology only identifies that the voice source is the same user, but it cannot determine that "turn down the volume" is a valid command while "What do you think of the plot?" is casual conversation. At this point, the ASR confidence threshold mechanism fails because the text confidence of both audio segments is at a moderate level, and the target speaker binding mechanism cannot eliminate non-target speech interference generated by the user. As a result, the system may incorrectly execute semantic instructions in the chat content or miss real instructions, leading to functional failure, repeated user operations, and blocked interaction processes.

[0024] If this technical issue persists, the false trigger rate of voice assistants in open environments will increase significantly. Continuous processing of non-target speech will consume computational resources from audio decoding, semantic parsing, and action execution modules. System dialogue state management will suffer from slot filling errors due to the incorporation of invalid speech fragments, causing subsequent interaction logic to deviate from the user's true intent. Furthermore, frequent misoperations will force users to actively intervene in the system state, disrupting the natural continuity of voice interaction and ultimately reducing user trust in and willingness to use the voice assistant.

[0025] Based on this, embodiments of this application provide a voice interaction method, electronic device, and storage medium. According to the playback status information, dialogue status information, and multi-dimensional audio features in the collected streaming audio data of the voice assistant, it can determine whether the current audio contains an interactive intent directed towards the voice assistant. In complex scenarios where users may issue commands to the assistant or chat with others, it can accurately determine whether the voice is directed towards the voice assistant, thereby reducing the false triggering of non-interactive voice and the missed recognition of valid commands.

[0026] See Figure 1 In one embodiment, a voice interaction method is provided, which includes, but is not limited to, steps S101 to S104.

[0027] Step S101: Obtain the playback status information, dialogue status information, and collected streaming audio data of the voice assistant.

[0028] Playback status information refers to the current audio playback status of the voice assistant. For example, this information can indicate that the voice assistant is playing music, news, podcasts, or is in a mute or paused state.

[0029] Dialogue status information refers to the progress or stage of the current conversation between the voice assistant and the user. For example, this information may indicate that the voice assistant is waiting for user input, asking a question, confirming an operation, or that the conversation has ended.

[0030] Streaming audio data refers to continuous audio signals acquired in real time through acquisition devices such as microphones.

[0031] Voice assistants can internally maintain their current playback status, for example, by querying the running status of their audio playback module. Simultaneously, the voice assistant's dialogue management module can provide real-time dialogue status, indicating whether it is waiting for a user response or performing a task. Streaming audio data can be continuously acquired through the voice assistant's built-in microphone array and converted into a data stream.

[0032] Step S102: Based on the streaming audio data, combined with playback status information and dialogue status information, feature extraction is performed to obtain multi-dimensional audio features.

[0033] Multidimensional audio features include acoustic interference features, text confidence features, dialogue state features, and semantic orientation features. In essence, multidimensional audio features refer to a set of features extracted from streaming audio data that represent the audio content and environment across multiple dimensions.

[0034] Acoustic interference features refer to characteristics that reflect non-target speech components or environmental noise in streaming audio data. For example, these features may include the intensity of background noise, the presence of echoes, the proportion of music or external media sounds, and are used to assess speech intelligibility and environmental complexity.

[0035] Text confidence features refer to features that reflect the reliability of a speech recognition system's recognition results on streaming audio data. For example, these features may include the overall confidence level of the recognized text, the degree of ambiguity during the recognition process, etc., and are used to evaluate the accuracy of the recognition results.

[0036] Dialogue state features refer to the features extracted from dialogue state information that characterize the current dialogue context. For example, these features may include the type of content currently being played by the voice assistant, the type of slot waiting to be filled by the user, etc., to help determine whether the user's voice intent is relevant to the current dialogue.

[0037] Semantic targeting features refer to features that reflect whether the semantic content contained in streaming audio data points to a voice assistant. For example, these features may include identifying whether the text contains wake words, command words, question words, or casual conversation content, which are used to directly determine the user's voice interaction target.

[0038] Acoustic interference features can be obtained by identifying the presence and intensity of background noise through energy detection and spectral analysis of streaming audio data. Text confidence features can be characterized by the overall recognition confidence of the recognized text output by the speech recognition system. Dialogue state features can be provided by the dialogue management module as an indication of dialogue activity or inactivity. Semantic pointing features can be detected by a preset keyword matching module to determine whether the recognized text contains specific wake words or command words.

[0039] Step S103: Calculate the interaction direction score based on the multi-dimensional audio features.

[0040] Interaction orientation score is used to characterize whether streaming audio data has an interactive intent towards a voice assistant. In essence, the interaction orientation score is a numerical value calculated based on multi-dimensional audio features, used to quantify the likelihood that streaming audio data has an interactive intent towards a voice assistant. A higher interaction orientation score indicates that the user's voice is more likely to be interacting with a voice assistant.

[0041] The interaction direction score is calculated based on multi-dimensional audio features. Extracted acoustic interference features, text confidence features, dialogue state features, and semantic direction features are linearly weighted and summed using preset weights to obtain a comprehensive score. For example, the score will be higher when acoustic interference is low, text confidence is high, and a wake word is included.

[0042] Step S104: When the interactive pointing score meets the preset rejection conditions, the streaming audio data is rejected.

[0043] Rejection criteria refer to preset standards used to determine whether to reject streaming audio data. For example, when the interaction score is lower than a certain preset threshold, the rejection criteria are met.

[0044] Before performing the rejection operation, a fixed scoring threshold can be preset as the rejection condition. When the calculated interaction direction score is lower than the threshold, it is determined that the streaming audio data does not have an interaction intent towards the voice assistant, and thus it is rejected, and does not enter the subsequent intent execution process or trigger the action execution.

[0045] The following example will provide a more detailed explanation of the above technical solution: Imagine a family setting where user A is having a casual conversation with another family member while a voice assistant in the room is playing background music. User A might be chatting with family members or issuing commands to the voice assistant. Existing technologies face challenges in distinguishing between these scenarios; for example, speaker separation technology alone cannot determine whether user A's voice is directed at the voice assistant or at another person.

[0046] The voice interaction method in this embodiment can effectively solve this problem. When user A chats with a family member, for example, user A says "The weather is really nice today," the voice assistant will continuously collect the streaming audio data. At the same time, the voice assistant obtains its current playback status information (playing background music) and dialogue status information (idle waiting for instructions).

[0047] Feature extraction was performed on the streaming audio data. Due to the presence of background music, the extracted acoustic interference features may show a high probability of music sounds and low speech activity detection information. The overall confidence score of the recognized text "The weather is really nice today" may be low due to background music interference. The dialogue state feature indicates that the voice assistant is in an inactive dialogue state. Semantic pointing feature analysis revealed that the recognized text does not contain any wake words or command words.

[0048] Interaction direction scores are calculated based on these multidimensional audio features. Due to significant acoustic interference, low text confidence, and a lack of clear semantic direction features, the calculated interaction direction scores will be relatively low.

[0049] When the interaction points to a score below a preset rejection criterion (e.g., a low score threshold), the voice assistant will reject the streaming audio data. This means the voice assistant will not process "The weather is really nice today" as a command, thus avoiding false responses.

[0050] If user A then turns to the voice assistant and clearly says, "Xiao Ai, pause the music," the voice assistant will also collect the streaming audio data. The system obtains playback status information (music is playing) and dialogue status information (idle). At this point, the extracted multidimensional audio features will differ. Acoustic interference features may show that user A's speech energy is high, improving the signal-to-noise ratio. The overall confidence in the text confidence features will be increased due to the presence of clear speech and the wake word. Semantic pointing features will clearly detect the wake word "Xiao Ai" and the command "pause the music."

[0051] Based on these characteristics, the calculated interaction direction score will be significantly improved and exceed the preset rejection threshold. At this point, the voice assistant will not reject the streaming audio data, but will perform subsequent speech recognition and intent understanding processing, and finally execute the "pause music playback" command.

[0052] As can be seen from the above examples, the voice interaction method in this embodiment can effectively distinguish the user's voice interaction intent by comprehensively analyzing multi-dimensional audio features. Compared with existing methods that rely solely on speaker separation or single ASR confidence judgment, this method can more comprehensively consider multiple dimensions such as acoustic environment, recognition reliability, dialogue context, and semantic orientation. This allows for accurate determination of whether the voice is directed at the voice assistant in complex scenarios where the user may be giving commands to the assistant or engaging in casual conversation with others. This method avoids the misidentification and misresponse problems caused by the inability to determine whether "this sentence is addressed to the assistant" in existing technologies, significantly improving the interaction accuracy of the voice assistant and the user experience.

[0053] See Figure 2 In one embodiment, the specific method of step S102 includes, but is not limited to, steps S201 to S203.

[0054] Step S201: Based on the playback status information, perform acoustic feature recognition on the streaming audio data to extract interfering audio from the streaming audio data and obtain acoustic interference features.

[0055] Acoustic feature recognition of streaming audio data can be achieved by employing deep learning-based acoustic models, such as convolutional neural networks or recurrent neural networks, to classify and regress audio signals to identify different types of acoustic events and parameters. Alternatively, signal processing methods, such as short-time Fourier transform and Mel-frequency cepstral coefficients, can be used to extract time-frequency features of the audio, followed by pattern recognition using support vector machines or Gaussian mixture models. Then, sound source separation techniques, such as independent component analysis or non-negative matrix factorization, can be used to decompose the mixed audio signal into multiple independent sound sources, thereby separating out interference sources. Alternatively, a pre-trained acoustic event detection model can be used to identify and label non-speech events in the audio stream, such as music and noise, and remove them from the valid speech to obtain acoustic interference features.

[0056] In some embodiments, acoustic interference features may include speech activity detection information, signal-to-noise ratio (SNR), echo remnant ratio, probability of musical sound, probability of external media sound, probability of speech overlap, probability of near-field speech, source direction stability, and audio energy variation characteristics. Speech activity detection information indicates the presence of speech activity in the audio stream and its start and end times, and can be predicted based on a deep learning-based VAD model. SNR quantifies the intensity of the speech signal relative to background noise and can be estimated in real-time using algorithms such as spectral subtraction or Wiener filtering. Echo remnant ratio indicates the presence and intensity of incompletely eliminated echo components in the audio, and can be obtained by comparing the residual energy of a reference signal and the signal after echo removal. Probability of musical sound, probability of external media sound, and probability of speech overlap represent the likelihood that the audio contains music, external media sound, or multiple speech sources emitting sound simultaneously, respectively, and can be obtained by training corresponding classifiers or using source separation techniques. Near-field speech probability indicates the likelihood that a speech source is close to the microphone and can be obtained based on source localization techniques or by analyzing the energy attenuation characteristics of the speech signal. Sound source direction stability describes the degree of change in the direction of a speech source over a period of time. It can be determined by continuously calculating the sound source direction and evaluating its variance. Audio energy variation characteristics describe the pattern of audio signal energy change over time. They can be determined by calculating the short-time energy of the audio signal and analyzing its derivative.

[0057] Step S202: Confidence features and semantic features are extracted from the identified text obtained from the identified streaming audio data to obtain text confidence features and semantic orientation features.

[0058] Confidence feature extraction from the recognized text can be achieved by directly obtaining confidence scores from the decoder output of the ASR model, or by combining scores from the language model and acoustic model and calculating more refined confidence scores through post-processing algorithms. Semantic feature extraction from the recognized text can be performed using deep learning-based natural language understanding models, such as BERT and GPT, to perform intent classification and slot filling, or by utilizing traditional NLU methods such as keyword matching, rule engines, and semantic networks.

[0059] In some embodiments, text confidence features may include sentence-level confidence, word-level confidence, N-best candidate uncertainty, text fragment stability, and out-of-vocabulary (OV) probability. Semantic pointing features include at least the probability that the identified text contains an executable intent, the probability that it contains casual conversation, and the probability that it contains interruptions. Word-level confidence represents the accuracy probability of each word in the identified text, and can be the posterior probability output by the ASR decoder when generating each word. N-best candidate uncertainty evaluates the degree of difference between multiple best identification candidate results generated by the ASR system, and can be obtained by calculating the edit distance or semantic similarity between different candidates in the N-best list. Text fragment stability describes the stability of the identified text fragment at different time points in continuous streaming audio input, and can be obtained by comparing the consistency of multiple identification results of the same audio segment within a sliding window. OV probability represents the probability that the identified text contains words not found in the ASR system dictionary, and can be obtained by checking whether the words in the identified text are found in a preset dictionary. Semantic pointing features determine whether the user's intent is directed towards the voice assistant. For example, the probability of containing an executable intent represents the likelihood that the identified text contains instructions or requests that the voice assistant can understand and execute. This can be determined by training an intent classification model or building a database of intent keywords and phrases. The probability of containing casual conversation represents the likelihood that the identified text contains non-instructional, non-task-related casual conversation or greetings. This can be detected by training a casual conversation detection model. The probability of containing interruptions represents the likelihood that the identified text contains expressions indicating that the user is attempting to interrupt the voice assistant's current speech or operation. This can be detected by building a list of interruption keywords and phrases.

[0060] Step S203: Extract state features from the dialogue state information to obtain dialogue state features.

[0061] State feature extraction of dialogue state information can be performed by encoding the dialogue state information into a vector representation, such as one-hot encoding or embedding vectors, or by directly extracting structured features from the dialogue state.

[0062] In some embodiments, dialogue state features may include the voice assistant's broadcast status and the type of slot to be filled. Dialogue state features describe the current dialogue context of the voice assistant. For example, the voice assistant's broadcast status indicates whether the voice assistant is currently broadcasting a message; this could be a flag in the voice assistant's internal state machine. The type of slot to be filled indicates what type of information the voice assistant is waiting for from the user to fill a slot in the current dialogue task; this could be a field in the current task state maintained by the dialogue management module.

[0063] The voice interaction method of this application, after acquiring the playback status information and dialogue status information of the voice assistant, as well as the collected streaming audio data, performs detailed feature extraction on the streaming audio data to more accurately determine the user's interaction intent. Specifically, the method first performs acoustic feature recognition on the streaming audio data based on the playback status information, thereby effectively separating interfering audio from the complex acoustic environment and quantifying it into acoustic interference features. These features, such as speech activity detection information, signal-to-noise ratio, echo persistence ratio, probability of music sound, probability of external media sound, probability of speech overlap, probability of near-field speech, source direction stability, and audio energy change features, together depict the quality and environmental background of the current speech signal, providing important environmental context for subsequent speech recognition and intent determination. Simultaneously, confidence features and semantic features are extracted from the recognized text obtained after speech recognition. Text confidence features, including whole-sentence confidence, word-level confidence, N-best candidate uncertainty, text segment stability, and out-of-vocabulary word probability, are used to evaluate the reliability of the speech recognition results and identify potential recognition errors or uncertainties. Semantic orientation features, such as the probability of containing an executable intent, the probability of containing casual conversation, and the probability of containing an interruption, directly reveal the potential intent type of the user's speech at the text content level. Furthermore, this method extracts state features from the voice assistant's dialogue state information, obtaining dialogue state features such as the voice assistant's playback state and the type of slot to be filled. These dialogue state features provide crucial contextual information, enabling the system to understand the dialogue context in which the user's speech is generated, such as whether the user interrupts during the voice assistant's playback or provides input while waiting for specific information. Through the above mechanisms, the solution in this application organically combines information from four dimensions: acoustic environment, speech recognition quality, textual semantic intent, and dialogue context, improving the accuracy of interaction orientation judgment.

[0064] See Figure 3 In one embodiment, the specific method of step S103 includes, but is not limited to, steps S301 to S302.

[0065] Step S301: The acoustic interference features, text confidence features, dialogue state features, and semantic orientation features are fused to obtain multi-dimensional fused features.

[0066] Step S302: Input the multi-dimensional fusion features into the scoring model and obtain the output interactive pointing score.

[0067] This application's solution achieves interaction-oriented scoring by integrating multi-dimensional audio features and utilizing a scoring model for determination. First, acoustic interference features, text confidence features, dialogue state features, and semantic orientation features are fused to generate a unified, higher-information-density multi-dimensional fused feature. This ensures all relevant information is integrated into a coherent representation, avoiding information fragmentation and providing comprehensive input for subsequent decision-making. Subsequently, the multi-dimensional fused feature is input into a pre-trained scoring model. This scoring model, through learning from extensive historical data, has mastered the complex mapping relationship between multi-dimensional fused features and user interaction intent. Upon receiving the multi-dimensional fused feature, the scoring model can output a quantified interaction-oriented score based on its internally learned patterns. This score intuitively represents the potential for the current streaming audio data to have an interaction intent oriented towards a voice assistant.

[0068] In one specific embodiment, the expression for generating the interaction-oriented rating is: , in, For interactive rating, Acoustic interference characteristics, For text confidence features, Features of the dialogue state For semantic pointing features, For feature fusion operations, For sigmoid activation operation, and These are the model parameters.

[0069] In some embodiments, the voice interaction method further includes: performing error correction processing on the recognized text obtained from the recognized streaming audio data when the interaction pointing score does not meet the rejection condition and the text confidence feature does not meet the preset confidence condition.

[0070] This application's solution initiates timely error correction when a user's interaction intent is initially determined but the recognized text's reliability is insufficient, avoiding misunderstandings of user intent or unnecessary rejection due to recognition errors. Specifically, in the voice interaction method, features are first extracted based on streaming audio data, playback status information, and dialogue status information to obtain multi-dimensional audio features. An interaction direction score is then calculated to initially determine whether the user has an interaction intent with the voice assistant. When the interaction direction score indicates that the streaming audio data should not be rejected, i.e., an interaction intent may exist, the system further evaluates the reliability of the recognized text. If, at this point, the text confidence feature shows insufficient confidence in the recognized text, i.e., the recognition result may be erroneous or uncertain, this method does not immediately execute subsequent operations based on the recognized text, but instead initiates an error correction process. In this way, it avoids directly rejecting user input or executing incorrect operations when interaction intent may exist but the recognized text is inaccurate, thereby improving the accuracy of voice interaction and user experience. This mechanism provides an intermediate step when a user's interaction intent is initially determined but the recognition result is questionable, ensuring rigor in processing user commands.

[0071] The following example illustrates this. When a user says "Play Jay Chou's songs," the system first acquires streaming audio data and combines it with current playback status information (e.g., the voice assistant is not currently playing music) and dialogue status information (e.g., there is no dialogue involving a specific task) to extract features. Assume the calculated interaction direction score indicates the user has a clear interaction intent, thus not meeting the rejection criteria. However, if the word-level confidence score of the identified text "Play Jay Chou's songs" is low, causing the overall text confidence feature to fail to meet the preset confidence criteria, the system will initiate error correction processing for the identified text. For example, the system might correct "Jay Chou" to "Jay Chou" based on pronunciation similarity, user history preferences, or a list of popular singers, or prompt the user for confirmation when unsure.

[0072] See Figure 4 In one embodiment, the method for error correction processing of the identified text obtained from the identification of streaming audio data includes, but is not limited to, steps S401 to S402.

[0073] Step S401: Extract word fragments from the recognized text whose word-level confidence is lower than a preset word-level confidence threshold.

[0074] Step S402: Generate at least one error correction candidate for word fragment based on at least one of the following: N-best candidate of the identified text, phonetic similar words, domain entity library, current dialogue context and current task state.

[0075] At least one correction candidate for the generated word segment can be selected from the N-best candidate list provided by the speech recognition system, choosing a candidate word that corresponds to the word segment to be corrected in terms of position. Simultaneously, words with similar pronunciations are generated based on the pinyin or pronunciation of the word segment to be corrected, using a phonetic dictionary or phonological model. Furthermore, words highly relevant to the current dialogue context (e.g., what the user has said before, content already played by the system) and the current task status (e.g., the type of task being performed, information about filled slots) can be retrieved from a predefined domain entity library as correction candidates.

[0076] Step S403: Calculate the comprehensive score of the error correction candidates, and select the error correction candidate with the highest comprehensive score as the target error correction candidate.

[0077] The overall score can be achieved using a weighted sum model, which linearly or non-linearly combines factors such as pronunciation similarity, contextual matching, prior probability, and editing cost, assigning different weights to them. Alternatively, machine learning models (such as support vector machines or neural networks) can be used to train these features and output a score representing the probability of a candidate word being correct.

[0078] The overall score is determined based on at least one of the following: the pronunciation similarity between the error correction candidate and the word segment, the matching degree between the error correction candidate and the current dialogue context, the prior probability of the error correction candidate in its domain, and the editing cost required to modify it into an error correction candidate.

[0079] Step S404: When the overall score is higher than the preset score threshold and the action risk level of the action corresponding to the target error correction candidate is lower than the preset risk threshold, execute the action corresponding to the target error correction candidate.

[0080] Action risk levels can be classified according to the nature of the action. For example, actions involving querying information have lower risk, while actions involving payment or modification of important settings have higher risk.

[0081] The risk threshold can be dynamically adjusted based on the specific application scenario and the user's risk tolerance.

[0082] Step S405: When the overall score is not higher than the score threshold, or the action risk level of the action corresponding to the target error correction candidate is not lower than the risk threshold, listen to the expression in the candidate listening set within the preset listening window.

[0083] The candidate audiometry set includes at least the target error correction candidate and the corresponding confirmation expression.

[0084] In voice interaction, when the system extracts features and calculates an interaction direction score based on streaming audio data, playback status information, and dialogue status information, and determines that the user's intent is directed towards the voice assistant, but the confidence level of the recognized text is low, the solution proposed in this application aims to improve the accuracy of the interaction and the user experience by using a refined error correction mechanism to avoid directly executing incorrect instructions or simply rejecting recognition. The solution first identifies word segments in the recognized text that the speech recognition system considers to have low accuracy. These word segments are potential error sources and require focused attention. To correct these potential errors, the system comprehensively utilizes multiple information sources, including the N-best candidates from speech recognition, words with similar pronunciations, predefined domain entity knowledge, and the current dialogue context and task state, to generate a series of possible error correction candidates. These candidate words provide alternatives to the original low-confidence word segments from different dimensions. Subsequently, the system comprehensively evaluates these generated error correction candidates and calculates a comprehensive score for each candidate. This comprehensive score calculation considers the pronunciation similarity between the error correction candidate and the original word segment, the degree of matching with the current dialogue context, the prior probability in a specific domain, and the editing cost required to modify the original word segment into an error correction candidate. Through multi-dimensional evaluation, the system can more accurately determine which error correction candidate is most likely to represent the user's true intent. The candidate with the highest score is selected as the target error correction candidate. After obtaining the target error correction candidate, the system performs a dual judgment: first, it judges whether the overall score of the target error correction candidate is high enough to ensure its reliability; second, it judges whether the risk level of the action corresponding to the target error correction candidate is within an acceptable range. Only when the error correction result is highly reliable and the execution risk is low will the system automatically execute the action corresponding to the target error correction candidate. This mechanism effectively avoids misoperations caused by incorrect error correction and ensures the security of the interaction. However, if the overall score of the target error correction candidate is not high enough, or its corresponding action has a high risk, the system will not execute it rashly. Instead, it will enter a preset listening window, during which the system will pay special attention to whether the user speaks a voice that matches the target error correction candidate or its confirmation expression. By listening to user feedback, the system provides the user with an opportunity to confirm or clarify, thereby maintaining the smoothness and accuracy of the interaction even in uncertain situations, avoiding the user frustration that may be caused by simple rejection, and providing the system with opportunities for further learning and optimization.

[0085] The following example illustrates this. A user says to the voice assistant, "Play Jay Chou's songs," but due to environmental noise or inaccurate pronunciation, the speech recognition system identifies "Jay Chou" as "Zhou Jieren." At this point, the system detects that the word-level confidence score of "Zhou Jieren" is lower than the preset word-level confidence threshold. The system then generates error correction candidates for the word fragment "Zhou Jieren." For example, it might find "Jay Chou" from the N-best candidates, find "Jay Chou" and "Zhou Jieren" from a phonetic similarity dictionary, and combine this with the current dialogue context (the user may frequently listen to music) and the domain entity library (a list of popular singers) to generate candidates such as "Jay Chou" and "JJ Lin." Next, the system calculates the comprehensive score of these error correction candidates. For example, for "Jay Chou," its pronunciation similarity to "Jay Chou" is very high, its contextual match with "play...songs" is high, its prior probability in the popular music domain is high, and its editing cost is low; therefore, its comprehensive score will be very high. While "Lin Junjie" is also a singer, his pronunciation is less similar to "Zhou Jielun," resulting in a relatively lower overall score. Let's assume "Zhou Jielun" is selected as the target error correction candidate. The system then determines if "Zhou Jielun's" overall score exceeds a preset score threshold. If it does, and the risk level of playing the song (usually low) is below the preset risk threshold, the system will execute the action of "playing Zhou Jielun's song." However, if the recognition result is "playing Zhou Jielun's song," and while "Zhou Jielun's" overall score is the highest, it still doesn't reach the preset score threshold (e.g., the system still has significant doubts about recognizing "Zhou Jielun"), or if the action of playing the song is set to high risk (e.g., requiring payment), the system will not execute it directly. In this case, the system will listen to the user's possible responses within a preset listening window. For example, the system might prompt, "Do you want to play Zhou Jielun's song?" and listen for confirmation responses like "yes" or "correct," or the user might directly repeat "Zhou Jielun" for correction. If the user says "yes" in the listening window, the system will play a Jay Chou song.

[0086] In some embodiments, before generating error correction candidates for word segments, the method further includes: determining whether the identified text contains an interruption intent based on semantic orientation features; when an interruption intent is detected, marking the temporary boundary between the foreground speech and the background speech based on the source direction stability and audio energy change features of the streaming audio data at the time corresponding to the interruption intent, and determining the acoustic feature template of the foreground speech; the acoustic feature template is used to perform micro-segment filtering on the streaming audio data after the interruption intent.

[0087] An interruption intent refers to a user's intention to interrupt the current task and start a new task by issuing a voice command while the voice assistant is broadcasting or playing other media.

[0088] An acoustic feature template for foreground speech is a set or model of acoustic features of the foreground speech, used to characterize its unique acoustic properties. Its purpose is to provide a reference benchmark for subsequent micro-segment selection, ensuring that the system can accurately identify and process subsequent audio data belonging to the user's foreground speech after interruption. For example, this template can include statistical models or mean-variance of features such as Mel-frequency cepstral coefficients (MFCC), linear predictive coding (LPC) coefficients, fundamental frequency (F0) information, energy, and speech rate. Another implementation is to encode the foreground speech using a deep learning model, generating a low-dimensional embedding vector as its acoustic feature template.

[0089] In the aforementioned voice interaction method, to more effectively handle interruption commands issued by users during voice assistant broadcasts, this application further introduces a mechanism for recognizing and processing interruption intentions. Specifically, the system first determines whether the recognized text contains an interruption intention based on its semantic orientation features. When an interruption intention is detected, the system does not immediately perform error correction. Instead, it utilizes the source direction stability and audio energy change features of the streaming audio data at the interruption moment to accurately mark the temporary boundary between the user's foreground speech and background speech. Determining this boundary is crucial because it distinguishes the valid command issued by the user from the voice assistant's own broadcasts or environmental noise. Subsequently, the acoustic feature template of the foreground speech is determined. This template serves as the acoustic fingerprint of the user's interruption command, guiding subsequent micro-segment selection of streaming audio data. In this way, even in complex acoustic environments, the system can accurately identify and extract speech segments belonging to the user's command after the user issues an interruption command, thereby avoiding misidentification of background sounds as user commands or omission of subsequent parts of the user's command, providing a cleaner and more accurate input for subsequent error correction or command execution. This approach enables voice assistants to respond more intelligently and robustly when interrupted by users, significantly improving the smoothness and accuracy of the interaction.

[0090] The following example illustrates this. When a voice assistant is broadcasting weather information, the user suddenly says, "Stop playing, play music." At this moment, the system first analyzes the semantic orientation features of the recognized text "Stop playing, play music." For example, using a pre-trained intent classification model, it determines that it contains a clear interruption intent. Once the interruption intent is detected, the system immediately analyzes the streaming audio data at the moment the user says "Stop playing." Specifically, the system can use a microphone array to locate the sound source and calculate the stability of the sound source direction at that moment, while simultaneously analyzing short-term energy changes in the audio signal. If the sound source direction is stable and the audio energy increases significantly, the system will mark a temporary boundary between the user's speech (foreground speech) and the voice assistant's broadcast (background speech). For example, at the starting point when the user says "stop," the system can determine a timestamp as the boundary. Next, the system extracts and determines the Mel-frequency cepstral coefficients (MFCCs) of the foreground speech "Stop playing" as its acoustic feature template. Subsequently, when the user continues to say "play music", the system will use the previously determined MFCC template to perform micro-segment filtering on the subsequent streaming audio data, extracting only audio segments that highly match the acoustic features of the template, thereby ensuring that the "play music" command can be accurately recognized and processed without being interfered with by the residual broadcast that the voice assistant may still be broadcasting or by environmental noise.

[0091] In some embodiments, before generating error correction candidates for word segments, the method further includes: determining the slot type of the current slot to be filled based on the current dialogue state of the voice assistant; establishing an error correction candidate whitelist based on the slot type; the error correction candidate whitelist contains words that match the slot type; filtering the N-best candidates of the recognized text using the error correction candidate whitelist, and using the filtered N-best candidates as the source of error correction candidates for word segments.

[0092] This application's solution determines the type of slot to be filled by utilizing the current dialogue state of the voice assistant and establishes a whitelist of error correction candidates with strong semantic relevance based on this slot type. This whitelist is then used to filter the N-best candidates for the recognized text, thereby limiting the scope of error correction candidates to a set of words highly relevant to the current dialogue context. This mechanism allows the error correction process to focus on the information the user truly intends to express, avoiding unnecessary error correction attempts on irrelevant words. In voice interaction methods, error correction is required when the interaction direction score does not meet the rejection condition and the text confidence feature does not meet the confidence condition. By introducing dialogue state and slot type filtering, the accuracy and efficiency of error correction can be significantly improved, enabling the entire voice interaction system to understand user intent more intelligently and accurately.

[0093] The description is given below through a specific example. Assume that a user is having a hotel booking conversation with a voice assistant, and the current dialogue state indicates that the voice assistant is waiting for the user to provide the information of "check-in date". The user utters "tomorrow", but due to ambient noise or inaccurate pronunciation, the speech recognition system recognizes "tomorrow" as a homophonic incorrect term, with a low word-level confidence. At this time, the voice assistant will determine that the type of the slot to be filled is "check-in date" according to the current dialogue state. The system will establish a correction candidate whitelist related to "check-in date", which may include date expressions such as "today", "tomorrow", "the day after tomorrow", "this Friday", "December 25, 2023" and the like. The N-best candidates generated by the speech recognition system for the incorrectly recognized term may include the incorrect term, "tomorrow", "pre-Ming" and the like. By filtering the N-best candidates against the "check-in date" whitelist, only the word "tomorrow" exists in both the N-best candidates and the whitelist, so "tomorrow" is selected as the final correction candidate source. The subsequent correction processing will be performed based on "tomorrow", for example, calculating its comprehensive score, and finally correcting the incorrectly recognized term to "tomorrow" In some embodiments, the voice interaction method further comprises: collecting subsequent behavior feedback information of the user on the rejection result; performing false rejection event identification according to the subsequent behavior feedback information to obtain a corresponding identification result; and updating the judgment threshold in the rejection condition according to the identification result, or taking the subsequent behavior feedback information as a training sample.

[0094] By introducing the user feedback mechanism, the solution of the present application enables the voice interaction method to have the capabilities of self-adaptation and continuous optimization. Specifically, after the system rejects the streaming audio data when the interaction orientation score calculated according to the multi-dimensional audio features meets the preset rejection condition, the system actively or passively collects the subsequent behavior feedback information of the user for this rejection behavior. These feedback information are then used to identify whether there is a false rejection event, so as to judge whether the previous rejection decision of the system is accurate. Based on the identification result of the false rejection event, the system can adopt two strategies for self-adjustment: one is to dynamically update the judgment threshold in the rejection condition, for example, when false rejection events occur frequently, appropriately relax the threshold to improve the command acceptance rate; the other is to take these subsequent behavior information with clear feedback as training samples for continuously optimizing the underlying model generating the interaction orientation score or the rejection decision model. Through this closed-loop feedback and learning mechanism, the system can continuously learn from actual user interactions and correct its own rejection deviation, thereby significantly improving the accuracy of the interaction orientation score and the rationality of the rejection strategy, enabling the voice assistant to understand the user's intention more accurately in a complex and changeable environment, reducing misoperations, and further improving the overall user experience.

[0095] The following example illustrates this. Suppose a user commands the voice assistant to "play Jay Chou's songs." However, due to significant ambient noise, the system, after extracting features from the streaming audio data, playback status information, and dialogue status information, calculates an interaction direction score below a preset rejection threshold, thus rejecting the streaming audio data and not executing the playback operation. At this point, the system collects the user's subsequent behavioral feedback information regarding the rejection result. For example, the user might immediately and clearly repeat the command "play Jay Chou's songs" again, or click the "retry" button through the voice assistant app interface. The system detects the user's repeated voice input or recognizes the user clicking the "retry" button, and based on this subsequent behavioral feedback information, identifies it as a false rejection event. Based on this identification result, the system can take any of the following measures: On the one hand, the system can dynamically adjust the judgment threshold in the rejection condition according to the frequency and severity of the false rejection event. For example, the rejection threshold for interaction-oriented scoring can be lowered from 0.7 to 0.65 to reduce the probability of rejection in similar scenarios in the future. On the other hand, the system can mark the streaming audio data that was falsely rejected and its corresponding multi-dimensional audio features, user repetition commands, etc., as training samples that "should be accepted but were rejected" and add them to the training dataset for retraining or fine-tuning the scoring model that generates interaction-oriented scoring. This will enable the model to more accurately judge the user's intent in subsequent interactions and avoid similar false rejections from happening again.

[0096] This application also provides an electronic device. (See appendix.) Figure 5 , Figure 5 This is a schematic diagram of the main structure of an electronic device according to an embodiment of this application. Figure 5 As shown, the electronic device in this embodiment mainly includes a processor 501 and a memory 502. The memory 502 can be configured to store a program for executing the methods of the above-described method embodiments, and the processor 501 can be configured to execute the program in the memory 502. This program includes, but is not limited to, a program for executing the methods of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application.

[0097] In some possible embodiments of this application, the electronic device may include multiple processors 501 and multiple memories 502. The program executing the methods of the above method embodiments may be divided into multiple subroutines, each of which may be loaded and run by a processor 501 to perform different steps of the methods of the above method embodiments. Specifically, each subroutine may be stored in a different memory 502, and each processor 501 may be configured to execute programs in one or more memories 502 to jointly implement the methods of the above method embodiments; that is, each processor 501 executes different steps of the methods of the above method embodiments to jointly implement the methods of the above method embodiments.

[0098] The aforementioned multiple processors 501 can be processors deployed on the same device. For example, the aforementioned electronic device can be a high-performance device composed of multiple processors, and the aforementioned multiple processors 501 can be processors configured on that high-performance device. Alternatively, the aforementioned multiple processors 501 can also be processors deployed on different devices. For example, the aforementioned electronic device can be a server cluster, and the aforementioned multiple processors 501 can be processors on different servers within the server cluster.

[0099] This application also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to this application, the computer-readable storage medium can be configured to store a program that performs the methods of the above-described method embodiments, the program of which can be loaded and run by a processor to implement the above-described methods. For ease of explanation, only the parts related to the embodiments of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of this application. The computer-readable storage medium can be a memory formed by various electronic devices. Optionally, in the embodiments of this application, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0100] The voice interaction method, electronic device, and storage medium provided in this application extract features based on the playback status information and dialogue status information of the voice assistant and the collected streaming audio data to obtain multi-dimensional audio features. By comprehensively analyzing the multi-dimensional audio features, considering multiple dimensions such as acoustic environment, recognition reliability, dialogue context, and semantic orientation, an interaction orientation score representing whether the streaming audio data has an interaction intention towards the voice assistant is calculated. When the interaction orientation score meets the preset rejection conditions, the streaming audio data is rejected. In complex scenarios where users may give instructions to the assistant or chat with others, it can accurately determine whether the voice is directed towards the voice assistant, reducing the false triggering of non-interactive voice and the missed recognition of valid instructions.

[0101] Exemplary embodiments of this disclosure have been specifically shown and described above. It should be understood that this disclosure is not limited to the detailed structures, arrangements, or implementations described herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A voice interaction method, characterized in that, include: Acquire playback status information, dialogue status information, and collected streaming audio data from the voice assistant; Based on the streaming audio data, combined with the playback status information and the dialogue status information, feature extraction is performed to obtain multi-dimensional audio features; The multidimensional audio features include acoustic interference features, text confidence features, dialogue state features, and semantic orientation features; Based on the multidimensional audio features, an interaction orientation score is calculated; the interaction orientation score is used to characterize whether the streaming audio data has an interactive intent toward the voice assistant. When the interactive pointing score meets the preset rejection conditions, the streaming audio data is rejected. When the interaction pointing score does not meet the rejection condition and the text confidence feature does not meet the preset confidence condition, the identified text obtained by recognizing the streaming audio data is subjected to error correction processing. The step of calculating the interaction-oriented score based on the multidimensional audio features includes: The acoustic interference features, the text confidence features, the dialogue state features, and the semantic orientation features are fused to obtain multi-dimensional fused features; Input the multidimensional fusion features into the scoring model to obtain the output interaction-oriented score; The error correction processing of the identified text obtained from the identification of the streaming audio data includes: Extract word fragments from the identified text whose word-level confidence is lower than a preset word-level confidence threshold; Based on at least one of the following: the N-best candidates of the identified text, phonetic similar words, domain entity library, current dialogue context, and current task state, at least one error correction candidate is generated for the word fragment. Calculate the comprehensive score of the error correction candidates, and select the error correction candidate with the highest comprehensive score as the target error correction candidate; the comprehensive score is determined based on at least one of the following: the pronunciation similarity between the error correction candidate and the word segment, the matching degree between the error correction candidate and the current dialogue context, the prior probability of the error correction candidate in its domain, and the editing cost required to modify it into the error correction candidate; When the overall score is higher than a preset score threshold and the action risk level of the action corresponding to the target error correction candidate is lower than a preset risk threshold, the action corresponding to the target error correction candidate is executed. When the overall score is not higher than the score threshold, or the action risk level of the action corresponding to the target error correction candidate is not lower than the risk threshold, the expression in the candidate audio set is tested within a preset audio testing window; the candidate audio set includes at least the target error correction candidate and the corresponding confirmation expression. Before generating the error correction candidates for the word fragments, the method further includes: Based on the semantic pointing features, determine whether the identified text contains an interruption intent; Upon detecting the interruption intention, based on the source direction stability and audio energy change characteristics of the streaming audio data at the time corresponding to the interruption intention, a temporary boundary between the foreground speech and the background speech is marked, and an acoustic feature template of the foreground speech is determined; the acoustic feature template is used to perform micro-segment filtering on the streaming audio data after the interruption intention.

2. The voice interaction method according to claim 1, characterized in that, The feature extraction based on the streaming audio data, combined with the playback status information and the dialogue status information, includes: Based on the playback status information, acoustic feature recognition is performed on the streaming audio data to extract interfering audio from the streaming audio data and obtain the acoustic interference features. Confidence features and semantic features are extracted from the identified text obtained by recognizing the streaming audio data to obtain the text confidence features and the semantic orientation features. The dialogue state information is subjected to state feature extraction to obtain the dialogue state features.

3. The voice interaction method according to claim 1, characterized in that, Before generating the error correction candidates for the word fragments, the method further includes: Based on the current dialogue state of the voice assistant, determine the slot type of the slot to be filled. Based on the slot type, a whitelist of error correction candidates is established; the whitelist of error correction candidates contains words that match the slot type; The N-best candidates of the identified text are filtered using the error correction candidate whitelist, and the filtered N-best candidates are used as the error correction candidate source for the word segment.

4. The voice interaction method according to claim 1, characterized in that, Also includes: Collect user feedback information on subsequent behaviors related to the rejection result; Based on the subsequent behavioral feedback information, false rejection events are identified to obtain the corresponding identification results; Based on the recognition result, update the judgment threshold in the rejection condition, or use the subsequent behavior feedback information as a training sample.

5. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the voice interaction method according to any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the voice interaction method according to any one of claims 1 to 4.

Images