A method, device and electronic device for voice interruption of intelligent body

Abstract

The embodiment of the present invention provides a method, device and electronic device for voice interruption of an intelligent agent, which relates to the field of data processing technology, wherein the method for voice interruption of an intelligent agent comprises: in the process of outputting response information, when the intelligent agent is in a listening state, obtaining the currently detected voice information; performing voice recognition on the voice information to obtain target text information; analyzing the target text information to determine whether the voice information belongs to interruption information; when the voice information belongs to interruption information, determining the interruption mode corresponding to the voice information; when the interruption mode corresponding to the voice information is interruptible, switching the intelligent agent from the listening state to the interruptible state; when the intelligent agent is in the interruptible state, when the preset interruption condition is met, interrupting the intelligent agent to output the response information. The embodiment of the present invention solves the problem that the intelligent agent cannot work normally due to voice interruption in a noisy environment.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a voice interruption method, device and electronic equipment for an intelligent body. Background Art

[0002] With the rapid development of language big models, the application of intelligent agents is becoming more and more extensive. Intelligent agents are a very important concept in the field of artificial intelligence. Any independent entity that can think and interact with the environment can be abstracted as an intelligent agent. Intelligent agents can specifically refer to software or hardware entities that can act autonomously. Human-computer interaction with intelligent agents can be carried out through interactive interfaces or through voice recognition. In order to minimize the interaction barriers between users and intelligent agents, it is very necessary to realize more anthropomorphic intelligent agents.

[0003] The voice interruption function is an important voice recognition technology that makes the intelligent agent more anthropomorphic. In the related technology, the voice interruption function is connected with the voice recognition module. When the intelligent agent detects voice input, it stops the current output (that is, interrupts the current output of the intelligent agent) and receives and responds to the current input of the user. In the case of a noisy environment around the intelligent agent, this voice interruption method will cause the intelligent agent to be interrupted all the time due to excessive noise, and thus cannot work normally. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device and electronic device for interrupting the voice of an intelligent agent, so as to solve the problem that the intelligent agent cannot work normally due to interruption of voice in a noisy environment. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for interrupting a voice of an intelligent agent, the method comprising:

[0006] In the process of outputting the response information, when the agent is in a listening state, the currently detected voice information is obtained;

[0007] Performing speech recognition on the speech information to obtain target text information;

[0008] Analyzing the target text information to determine whether the voice information is interruption information;

[0009] In the case where the voice information is an interruption information, determining an interruption mode corresponding to the voice information;

[0010] When the interruption mode corresponding to the voice information is interruptible, switching the agent from a listening state to an interruptible state;

[0011] When the agent is in an interruptible state, when a preset interruption condition is met, the agent is interrupted to output the response information.

[0012] Optionally, the method further comprises:

[0013] When the voice information does not belong to interruption information and there is no unoutput response information, monitoring the input voice information;

[0014] Alternatively, when the voice information does not belong to an interruption message and there is currently no response information output, the intelligent agent is switched from a listening state to an interruptible state, and the no-output response information is output; and when at least one complete response sentence in the no-output response information is output, the intelligent agent is switched from an interruptible state to a listening state to monitor the input voice information.

[0015] Optionally, the method further comprises:

[0016] When the interruption mode corresponding to the voice information is non-interruptible, switching the agent from a listening state to a non-interruptible state, and outputting the response information;

[0017] After all the response information is output, the agent is switched from an uninterruptible state to a listening state, and the input voice information is listened to.

[0018] Optionally, after switching the agent to a listening state, the method further comprises:

[0019] Determine whether there is no historical voice message being responded to;

[0020] If there is unresponded historical voice information and no voice information input is detected, responding to the unresponded historical voice information;

[0021] If there is no unresponded historical voice information, the input voice information is monitored and responded to.

[0022] Optionally, when a preset interruption condition is met, interrupting the agent from outputting the response information comprises:

[0023] When the agent pauses in outputting the response information, interrupting the agent from outputting the response information;

[0024] Alternatively, when the agent outputs at least one complete response sentence in the response information, the agent is interrupted from outputting the response information.

[0025] Optionally, analyzing the target text information to determine whether the voice information is interruption information includes:

[0026] The target text information and the historical text information are input into a pre-trained interruption filtering model for interruption analysis to obtain a result of whether the voice information belongs to interruption information; wherein the interruption filtering model is trained based on sample text information, historical sample text information, and the annotation result of whether the sample text information belongs to interruption information; the historical text information represents text information corresponding to historical voice information received within a historical time period before the moment when the voice information is detected.

[0027] Optionally, the determining an interruption mode corresponding to the voice information includes:

[0028] The target text information and the historical text information are input into a pre-trained state judgment model to judge the interruption method, and the interruption method corresponding to the voice information is obtained; wherein the state judgment model is trained based on sample text information, historical sample text information, and the marked interruption method corresponding to the sample text information; the historical text information represents the text information corresponding to the historical voice information received within the historical time period before the moment when the voice information is detected.

[0029] In a second aspect, an embodiment of the present invention provides a voice interruption device for an intelligent agent, the device comprising:

[0030] The voice acquisition module is used to acquire the currently detected voice information when the agent is in a listening state during the process of outputting the response information;

[0031] A speech recognition module, used for performing speech recognition on the speech information to obtain target text information;

[0032] An interruption filtering module, used to analyze the target text information to determine whether the voice information belongs to interruption information;

[0033] A mode determination module, used for determining an interruption mode corresponding to the voice information when the voice information belongs to interruption information;

[0034] A state switching module, used for switching the agent from a listening state to an interruptible state when the interruption mode corresponding to the voice information is interruptible;

[0035] The voice interruption module is used to interrupt the intelligent agent to output the response information when the preset interruption condition is met when the intelligent agent is in an interruptible state.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0037] Memory, used to store computer programs;

[0038] The processor is used to implement any of the above methods when executing the program stored in the memory.

[0039] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned methods is implemented.

[0040] In a fifth aspect, an embodiment of the present invention further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-described methods.

[0041] Beneficial effects of the embodiments of the present invention:

[0042] The embodiments of the present invention provide a method, device and electronic device for voice interruption of an intelligent agent. In the process of outputting response information, if the intelligent agent is in a listening state, the currently detected voice information is obtained, voice recognition is performed on the detected voice information, and the recognized target text information is interrupted and filtered, so as to achieve more intelligent interruption, effectively improve the conversation ability of the intelligent agent in a noisy environment, and when it is determined that the interruption mode corresponding to the voice information is interruptible, the intelligent agent is switched from the listening state to the interruptible state, and when the intelligent agent is in the interruptible state, the output of the response information of the intelligent agent is interrupted when the preset interruption condition is met, so that the intelligent agent can achieve more diverse and natural conversations, ensure the normal operation of the voice interruption function, and solve the problem that the intelligent agent cannot work normally due to voice interruption in a noisy environment.

[0043] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0045] Figure 1 A schematic diagram of a flow chart of a method for interrupting a voice of an intelligent agent provided by an embodiment of the present invention;

[0046] Figure 2 Another flow chart of a method for interrupting a voice of an intelligent agent provided by an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of a method for interrupting a voice of an intelligent agent provided by an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of a structure of a voice interruption device for an intelligent agent provided by an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0051] In order to solve the problem that voice interruption in a noisy environment makes an intelligent agent unable to work normally, an embodiment of the present invention provides a voice interruption method, device and electronic device for an intelligent agent. The intelligent agent corresponding to the voice interruption method for an intelligent agent provided in an embodiment of the present invention can be the intelligent agent itself, such as an intelligent robot, or a device or electronic device for deploying the intelligent agent, wherein the device can be, for example, a voice broadcast device, and the electronic device can be, for example, a client device or a server device.

[0052] like Figure 1 As shown, the voice interruption method for an intelligent agent provided by an embodiment of the present invention may include the following steps:

[0053] S101, in the process of outputting response information, when the agent is in a listening state, obtaining currently detected voice information;

[0054] S102, performing speech recognition on the speech information to obtain target text information;

[0055] S103, analyzing the target text information to determine whether the voice information is an interruption information;

[0056] S104, when the voice information is an interruption information, determining an interruption mode corresponding to the voice information;

[0057] S105, when the interruption mode corresponding to the voice information is interruptible, switching the agent from a listening state to an interruptible state;

[0058] S106, when the agent is in an interruptible state and a preset interruption condition is met, interrupt the agent to output a response message.

[0059] The embodiment of the present invention, during the process of outputting response information, if the intelligent agent is in a listening state, then the currently detected voice information is obtained, voice recognition is performed on the detected voice information, and the recognized target text information is interrupted and filtered, so as to achieve more intelligent interruption, effectively improve the conversation ability of the intelligent agent in a noisy environment, and, when it is determined that the interruption mode corresponding to the voice information is interruptible, the intelligent agent is switched from the listening state to the interruptible state, and when the intelligent agent is in the interruptible state, the output of the response information of the intelligent agent is interrupted when the preset interruption conditions are met, so that the intelligent agent can achieve more diverse and natural conversations, ensure the normal operation of the voice interruption function, and solve the problem that the voice interruption in a noisy environment makes the intelligent agent unable to work normally.

[0060] In the embodiment of the present invention, the voice interruption function is realized by the state transition of the intelligent agent, and the states of the intelligent agent include a monitoring state, an interruptible state, and an uninterruptible state. In the monitoring state, the intelligent agent monitors and responds to the input voice information; in the interruptible state, the voice interruption of the intelligent agent is realized; in the uninterruptible state, the intelligent agent does not respond to the input voice information and does not interrupt the output of the current response information.

[0061] like Figure 1 As shown, a voice interruption method for an intelligent agent according to an embodiment of the present invention includes:

[0062] S101, in the process of outputting response information, when the agent is in a listening state, obtaining currently detected voice information.

[0063] In the process of the intelligent agent outputting response information according to the set process or in response to the user's input, if the intelligent agent determines that it is in a monitoring state, it monitors the sound generated externally, and then obtains the currently detected voice information. Among them, the way in which the intelligent agent outputs the response information can be output in the form of text or in the form of voice broadcast, which is not limited by the embodiment of the present invention. The currently detected voice information is the sound generated externally during the process of the intelligent agent outputting the response information, including the sound input to the intelligent agent and other sounds in the surrounding environment. The way to obtain the currently detected voice information can be directly obtained by the intelligent agent, or it can be indirectly obtained through an external acquisition device, which is not limited by the embodiment of the present invention.

[0064] S102, performing speech recognition on the speech information to obtain target text information.

[0065] An Automatic Speech Recognition (ASR) model is deployed or installed in the intelligent agent. The ASR model can perform speech recognition on the speech information detected by the intelligent agent and can perform noise reduction on the recognized speech information during the speech recognition process to recognize the target text information in a readable text or text format that can be understood by the language model.

[0066] In one example, the ASR model can be FunASR, a large speech recognition model. FunASR is based on a deep learning algorithm that trains models to identify different features in speech signals and maps them to corresponding text. In order to better denoise the detected speech information, FunASR can be used with NVIDIA Broadcast (an application software). NVIDIA Broadcast can implement artificial intelligence (AI) functions such as microphone denoising, room echo cancellation, virtual background, portrait tracking, and video denoising to improve audio quality.

[0067] S103, analyzing the target text information to determine whether the voice information is interruption information.

[0068] In one example, the target text information can be analyzed by pre-set rules to determine whether the detected voice information belongs to interruption information. Exemplarily, the pre-set rules can be: when the target text information is a complete sentence, determine that the detected voice information belongs to interruption information, otherwise, determine that the detected voice information does not belong to interruption information; or, when the target text information is preset information, determine that the detected voice information belongs to interruption information, otherwise, determine that the detected voice information does not belong to interruption information, wherein the preset information can be a pre-set modal particle, interjection, etc. Of course, the pre-set rules can also be other pre-set contents, and the embodiments of the present invention do not give examples one by one here.

[0069] In a possible implementation, an interruption filtering model may be pre-trained, and then the target text information and historical text information may be input into the pre-trained interruption filtering model for interruption analysis to obtain the result of whether the voice information belongs to interruption information. The interruption filtering model is trained based on sample text information, historical sample text information, and the result of marking whether the sample text information belongs to interruption information; the historical text information represents the text information corresponding to the historical voice information received in the historical time period before the moment when the above voice information is detected. The duration corresponding to the historical time period may be set according to the actual situation, such as 5 seconds, 10 seconds or 1 minute, etc.

[0070] In one example, given that the Transformer-based bidirectional encoder representation (Bidirectional Encoder Representation from Transformers, BERT) pre-trained language model can better understand the relationship between the information input therein, the interruption filtering model used in an embodiment of the present invention is based on BERT and is fine-tuned and trained using sample text information in different dialogue scenarios, historical sample text information, and the annotation results of whether the sample text information belongs to interruption information.

[0071] Furthermore, the interruption filtering model obtained by training can better understand the relationship between the target text information and the historical text information input therein, and can judge the degree of association and / or logical accuracy of the target text information and the historical text information input therein. Further, according to the degree of association and / or logical accuracy of the target text information and the historical text information, it is judged whether the speech information corresponding to the target text information is noise or speech for interrupting the intelligent body, and the output of the interruption filtering model is obtained, that is, whether the speech information belongs to the interruption information.

[0072] Correspondingly, the interruption filtering model is a binary classification model. When training the interruption filtering model, binary cross entropy can be used as the loss function, which can be expressed as:

[0073]

[0074] in, represents the loss value, N represents the number of sample text information and historical sample text information, Indicates Whether the sample text information belongs to the interruption information can be indicated by 0 or 1. Indicates The probability that a sample text message is predicted to be an interruption message.

[0075] The detected voice information is interrupted and filtered using a pre-trained interruption filtering model. This is because the interruption filtering model can better understand the relationship between the target text information and the historical text information input therein, and can more accurately filter whether the detected voice information belongs to interruption information, thereby achieving more intelligent interruption and effectively improving the conversation ability of the intelligent agent in a noisy environment.

[0076] S104: When the voice information is interruption information, determine an interruption mode corresponding to the voice information.

[0077] When it is determined that the voice information is an interruption information, it means that there may be an interruption at present, and the importance of the voice information is further determined to determine the interruption mode corresponding to the voice information. In an example, the interruption mode may include interruptible and non-interruptable, and of course other modes may also be included, such as random interruption, conditional interruption, etc. Different interruption modes correspond to different states of the intelligent agent. Accordingly, determining the interruption mode corresponding to the voice information is equivalent to determining the state of the intelligent agent.

[0078] In a possible implementation, a state judgment model can be pre-trained, and then the target text information and historical text information can be input into the pre-trained state judgment model to judge the interruption method, and obtain the interruption method corresponding to the voice information. The state judgment model is trained based on sample text information, historical sample text information, and the annotated interruption method corresponding to the sample text information; the historical text information represents the text information corresponding to the historical voice information received in the historical time period before the moment when the above voice information is detected. The duration corresponding to the historical time period can be set according to the actual situation, such as 5 seconds, 10 seconds or 1 minute, etc.

[0079] In an embodiment of the present invention, the state judgment model can also be obtained by fine-tuning and training based on BERT as a basic model using sample text information, historical sample text information, and the annotation interruption method corresponding to the sample text information. Exemplarily, the sample text information and historical sample text information used in training the interruption filtering model can be the same as or different from the sample text information and historical sample text information used in training the state judgment model.

[0080] The trained state judgment model can judge the importance of the target text information according to the target text information and historical text information input therein, and then obtain the interruption mode corresponding to the target text information, that is, the interruption mode corresponding to the voice information. Exemplarily, taking the interruption mode including interruptible and non-interruptible as an example, when the state judgment model judges that the importance of the target text information is high, the interruption mode corresponding to the output target text information is non-interruptible, otherwise, the interruption mode corresponding to the output target text information is interruptible. Among them, interruptible and non-interruptible can be represented by 0 and 1 or other characters.

[0081] Of course, the interruption method may also include other methods. Accordingly, the state judgment model may be a multi-classification model. When training the state judgment model, the cross entropy may be used to calculate the loss function of the state judgment model. The loss function may be expressed as:

[0082]

[0083] in, represents the loss function, Indicates The probability that the interruption method corresponding to the sample text information is the selected annotation interruption method, Indicates The probability of the predicted interruption method corresponding to the sample text information, wherein the probability is 1 when the interruption method corresponding to the sample text information is the selected annotation interruption method, and 0 otherwise. The selected annotation interruption method can be any interruption method.

[0084] The pre-trained state judgment model is used to analyze the importance of the detected voice information to determine the corresponding interruption method of the voice information, realizing secondary interruption filtering of the voice information, enabling the intelligent agent to achieve more diverse and natural dialogue scenarios, and ensuring the normal operation of the voice interruption function.

[0085] S105, when the interruption mode corresponding to the voice information is interruptible, the agent is switched from the listening state to the interruptible state.

[0086] Different interruption modes correspond to different states of the agent. Furthermore, when it is determined that the interruption mode corresponding to the voice information is interruptible, the agent is switched from the listening state to the interruptible state, indicating that the current output response information can be interrupted.

[0087] S106, when the agent is in an interruptible state and a preset interruption condition is met, interrupt the agent to output a response message.

[0088] When the agent is in an interruptible state, in order to avoid abrupt interruption of the agent, the agent can be interrupted to output response information when the preset interruption conditions are met.

[0089] In a possible implementation, the preset interruption condition may be interruption when the output is paused, or interruption when a complete sentence is output, etc. Accordingly, the output of the response information by the agent may be interrupted when the agent pauses in outputting the response information; or the output of the response information by the agent may be interrupted when the agent outputs at least one complete response sentence in the response information.

[0090] Exemplarily, the pause when the agent outputs the response information may be when outputting in text form, a preset pause character may be output, such as a comma, a period, or an exclamation mark, or a preset pause word, etc. The pause when the agent outputs the response information may also be when outputting in voice broadcast form, a preset pause duration may be paused, such as 0.1 second, 0.5 second, or 1 second, etc.

[0091] When the agent is in an interruptible state, the agent can be interrupted when it pauses in outputting the response information or outputs at least one complete response sentence in the response information, so as to avoid the interruption of the agent being too abrupt.

[0092] like Figure 2 As shown, another voice interruption method for an intelligent agent according to an embodiment of the present invention includes:

[0093] S201, in the process of outputting response information, when the agent is in a listening state, obtaining the currently detected voice information.

[0094] S202, performing speech recognition on the speech information to obtain target text information.

[0095] S203, analyzing the target text information to determine whether the voice information is interruption information.

[0096] The specific implementation process of step S201 to step S203 may refer to the implementation process of the above-mentioned step S101 to step S103.

[0097] S204, when the voice information does not belong to interruption information and there is currently no unoutput response information, the input voice information is monitored; or, when the voice information does not belong to interruption information and there is currently no output response information, the intelligent agent is switched from the monitoring state to the interruptible state, and the unoutput response information is output; and when at least one complete response sentence in the unoutput response information is output, the intelligent agent is switched from the interruptible state to the monitoring state to monitor the input voice information.

[0098] When it is determined that the voice information does not belong to interruption information and there is currently no response information that has not been output, it means that there is currently no interruption, and the historical voice information or the response information of the last voice task has been output. At this time, the intelligent agent maintains the monitoring state unchanged and continues to monitor the input voice information.

[0099] When it is determined that the voice information does not belong to interruption information and there is currently no response information output, it means that there is no interruption at present, and the historical voice information or the response information of the last voice task is still being output. At this time, the intelligent agent is switched from the listening state to the interruptible state, and continues to output the non-output response information. When at least one complete response sentence in the non-output response information is output, the intelligent agent is switched from the interruptible state to the listening state, and the input voice information is listened to again.

[0100] According to an embodiment of the present invention, when the voice information does not belong to interruption information and there is currently no response information output, the intelligent agent is switched from a listening state to an interruptible state, and no response information is output; and when at least one complete response sentence in the no-output response information is output, the intelligent agent is switched from an interruptible state to a listening state, and the input voice information is monitored, thereby realizing the alternation of the listening state and the interruptible state of the intelligent agent, so as to realize the interruption of the intelligent agent more intelligently.

[0101] S205: When the voice information is interruption information, determine an interruption mode corresponding to the voice information.

[0102] S206, when the interruption mode corresponding to the voice information is interruptible, the agent is switched from the listening state to the interruptible state.

[0103] The specific implementation process of step S205 to step S206 may refer to the implementation process of the above-mentioned step S104 to step S105.

[0104] S207, when the preset interruption condition is met, the intelligent agent is interrupted to output a response message, and the intelligent agent is switched from an interruptible state to a listening state to listen to the input voice information.

[0105] When the agent is in an interruptible state, when the agent pauses in outputting response information, or when the agent outputs at least one complete response sentence in the response information, the agent outputs the response information, and further, the agent is switched from the interruptible state to the listening state to listen to the input voice information.

[0106] S208, when the interruption mode corresponding to the voice information is non-interruptible, the agent is switched from the listening state to the non-interruptible state, and a response message is output.

[0107] When the interruption mode corresponding to the voice information is not interruptible, it means that the current output response information is important and cannot be interrupted. At this time, the agent is switched from the listening state to the non-interruptible state, and the response information is continuously output. For example, the response information can be output in a streaming manner.

[0108] S209, after all the response information is output, the agent is switched from an uninterruptible state to a listening state, and the input voice information is listened to.

[0109] S210, determining whether there is any unresponded historical voice information.

[0110] After the agent is switched from the interruptible state or the non-interruptible state to the listening state, it can be further determined whether there is any unresponded historical voice information. The unresponded historical voice information can be, for example, the unresponded voice information heard by the agent when it was in the listening state last time, and the agent can cache the unresponded historical voice information.

[0111] S211: If there is unresponded historical voice information and no voice information input is detected, respond to the unresponded historical voice information.

[0112] Exemplarily, if there is unresponded historical voice information and no voice information input is currently detected, the unresponded historical voice information is responded to and the corresponding response information is output. If there is unresponded historical voice information and voice information input is currently detected, the currently detected voice information can be responded to first and the corresponding response information can be output. It is also possible to determine whether to respond to the currently detected voice information or the unresponded historical voice information and output the corresponding response information based on the preset priority or importance corresponding to the currently detected voice information and the unresponded historical voice information respectively. It is also possible to respond to the currently detected voice information and the unresponded historical voice information as a task and output the corresponding response information.

[0113] S212: If there is no unresponded historical voice information, the input voice information is monitored and responded to.

[0114] The embodiment of the present invention realizes the voice interruption function through the state transition of the intelligent body. Specifically, in the process of outputting response information, if the intelligent body is in the listening state, the currently detected voice information is obtained, the detected voice information is recognized, and the recognized target text information is interrupted and filtered to realize more intelligent interruption, effectively improving the conversation ability of the intelligent body in a noisy environment. Moreover, when it is determined that the interruption mode corresponding to the voice information is interruptible, the intelligent body is switched from the listening state to the interruptible state, and when the intelligent body is in the interruptible state, the output of the response information of the intelligent body is interrupted when the preset interruption condition is met, thereby expanding the state of the intelligent body, enabling the intelligent body to realize more diverse and natural conversations, ensuring the normal operation of the voice interruption function, and solving the problem that the intelligent body cannot work normally due to voice interruption in a noisy environment. Furthermore, when the voice information does not belong to interruption information and there is currently no response information output, the intelligent agent is switched from the listening state to the interruptible state, and the no-output response information is output. When at least one complete response sentence in the no-output response information is output, the intelligent agent is switched from the interruptible state to the listening state, and the input voice information is listened to, thereby realizing the alternation of the listening state and the interruptible state of the intelligent agent, so as to realize the interruption of the intelligent agent more intelligently.

[0115] Exemplarily, the voice interruption method for an intelligent agent provided by an embodiment of the present invention is as follows: Figure 3 As shown in the figure, when the agent is in the process of outputting the response information, the user's voice is recognized by the ASR model and the target text information is obtained ( Figure 3 Chinese text). Using the pre-trained interruption filtering model ( Figure 3 The target text information is analyzed by combining the historical text information with the encoding interruption filtering model in the target text information to obtain the result of whether the sound emitted by the user belongs to the interruption information. When the sound emitted by the user does not belong to the interruption information (i.e., no interruption) and there is currently no response information output (i.e., the previous task has not ended), the agent is switched from the listening state to the interruptible state, and the non-output response information is output, and when a complete response sentence in the non-output response information is output (i.e., a sentence ends), the agent is switched from the interruptible state to the listening state. When the sound emitted by the user does not belong to the interruption information and there is currently no non-output response information (i.e., no request), the agent maintains the listening state unchanged and monitors the input voice information. When the sound emitted by the user belongs to the interruption information (i.e., interruption), the pre-trained state judgment model is used in combination with the historical text information to analyze the importance of the target text information, and the interruption method corresponding to the target text information is obtained, that is, the interruption method corresponding to the sound emitted by the user. When the interruption mode is interruptible, it indicates that the current output response information is an interruptible task, and the intelligent agent is switched from the listening state to the interruptible state. When the intelligent agent pauses in outputting the response information, or when the intelligent agent outputs a complete response sentence in the response information, the intelligent agent outputs the response information. Furthermore, the intelligent agent is switched from the interruptible state to the listening state, and the input voice information is monitored. When the interruption mode is non-interruptible, it indicates that the current output response information is a non-interruptible task. The intelligent agent is switched from the listening state to the non-interruptible state, and continues to output the response information. After all the response information is output (i.e., the task is completed), the intelligent agent is switched from the non-interruptible state to the listening state, and the input voice information is monitored. In the embodiment of the present invention, the voice interruption function is realized through the state transition of the intelligent agent, which solves the problem that the intelligent agent cannot work normally due to voice interruption in a noisy environment.

[0116] Corresponding to the above method embodiments, the embodiments of the present invention also provide corresponding device embodiments.

[0117] like Figure 4 As shown, an embodiment of the present invention provides a voice interruption device for an intelligent agent, the device comprising:

[0118] The voice acquisition module 401 is used to acquire the currently detected voice information when the agent is in a listening state during the process of outputting the response information;

[0119] A speech recognition module 402 is used to perform speech recognition on speech information to obtain target text information;

[0120] An interruption filtering module 403 is used to analyze the target text information to determine whether the voice information is an interruption information;

[0121] A mode determination module 404, for determining an interruption mode corresponding to the voice information when the voice information is an interruption information;

[0122] The state switching module 405 is used to switch the agent from the listening state to the interruptible state when the interruption mode corresponding to the voice information is interruptible;

[0123] The voice interruption module 406 is used to interrupt the intelligent agent to output response information when the preset interruption condition is met when the intelligent agent is in an interruptible state.

[0124] Optionally, the above device further includes:

[0125] The first monitoring module is used to monitor the input voice information when the voice information does not belong to interruption information and there is currently no unoutput response information; or, when the voice information does not belong to interruption information and there is currently no output response information, the intelligent body is switched from the monitoring state to the interruptible state and outputs the unoutput response information; and when at least one complete response sentence in the unoutput response information is output, the intelligent body is switched from the interruptible state to the monitoring state to monitor the input voice information.

[0126] Optionally, the above device further includes:

[0127] A first response module, used for switching the agent from a listening state to an uninterruptible state and outputting a response message when the interruption mode corresponding to the voice information is uninterruptible;

[0128] The second monitoring module is used to switch the intelligent agent from an uninterruptible state to a monitoring state after all the response information is output, and monitor the input voice information.

[0129] Optionally, the above device further includes:

[0130] An information determination module, used to determine whether there is unresponded historical voice information;

[0131] A second response module, for responding to the unresponded historical voice information when there is unresponded historical voice information and no voice information input is detected;

[0132] The third monitoring module is used to monitor and respond to the input voice information when there is no unresponded historical voice information.

[0133] Optionally, the voice interruption module 406 is specifically used for:

[0134] When the intelligent agent pauses in outputting the response information, interrupt the intelligent agent from outputting the response information;

[0135] Alternatively, when the agent outputs at least one complete response sentence in the response information, the agent outputs the response information, and the response information is interrupted.

[0136] Optionally, the above-mentioned interruption filtering module 403 is specifically used to input the target text information and the historical text information into a pre-trained interruption filtering model to perform interruption analysis to obtain the result of whether the voice information belongs to interruption information; wherein the interruption filtering model is trained based on sample text information, historical sample text information, and the annotation result of whether the sample text information belongs to interruption information; the historical text information represents the text information corresponding to the historical voice information received in the historical time period before the moment when the voice information is detected.

[0137] Optionally, the above-mentioned method determination module 404 is specifically used to input the target text information and the historical text information into a pre-trained state judgment model to judge the interruption method when the voice information belongs to interruption information, so as to obtain the interruption method corresponding to the voice information; wherein the state judgment model is trained based on the sample text information, the historical sample text information, and the marked interruption method corresponding to the sample text information; the historical text information represents the text information corresponding to the historical voice information received in the historical time period before the moment when the voice information is detected.

[0138] The embodiment of the present invention further provides an electronic device, such as Figure 5 As shown, it includes a processor 501 , a communication interface 502 , a memory 503 and a communication bus 504 , wherein the processor 501 , the communication interface 502 , and the memory 503 communicate with each other via the communication bus 504 .

[0139] Memory 503, used for storing computer programs;

[0140] The processor 501 is used to implement any of the steps of the above-mentioned voice interruption method for an intelligent agent when executing the program stored in the memory 503 to achieve the same technical effect.

[0141] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0142] The communication interface is used for communication between the above electronic device and other devices.

[0143] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0144] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0145] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for voice interruption of an intelligent agent are implemented to achieve the same technical effect.

[0146] In another embodiment provided by the present invention, a computer program product containing instructions is also provided. When the computer is run on the computer, the computer executes the steps of any of the voice interruption methods for intelligent agents in the above embodiments to achieve the same technical effect.

[0147] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0148] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0149] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0150] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for interrupting a voice of an intelligent agent, characterized in that: The method comprises: In the process of outputting the response information, when the agent is in a listening state, the currently detected voice information is obtained; Performing speech recognition on the speech information using an automatic speech recognition (ASR) model to obtain target text information; Analyze the target text information using a pre-trained interruption filtering model to determine whether the voice information is interruption information; the interruption filtering model is trained based on the BERT model; In the case where the voice information is an interruption information, determining an interruption mode corresponding to the voice information; different interruption modes correspond to different states of the intelligent agent; When the interruption mode corresponding to the voice information is interruptible, switching the agent from a listening state to an interruptible state; When the agent is in an interruptible state, when the agent pauses in outputting the response information, interrupting the agent from outputting the response information; The method further comprises: When the voice information does not belong to interruption information and there is no unoutput response information, monitoring the input voice information; Alternatively, when the voice information does not belong to an interruption message and there is currently no response information output, the intelligent agent is switched from a listening state to an interruptible state, and the no-output response information is output; and when at least one complete response sentence in the no-output response information is output, the intelligent agent is switched from an interruptible state to a listening state to monitor the input voice information.

2. The method according to claim 1, characterized in that The method further comprises: When the interruption mode corresponding to the voice information is non-interruptible, switching the agent from a listening state to a non-interruptible state, and outputting the response information; After all the response information is output, the agent is switched from an uninterruptible state to a listening state, and the input voice information is listened to.

3. The method according to claim 2, characterized in that After switching the agent to the listening state, the method further includes: Determine whether there is no historical voice message being responded to; If there is unresponded historical voice information and no voice information input is detected, responding to the unresponded historical voice information; If there is no unresponded historical voice information, the input voice information is monitored and responded to.

4. The method according to claim 1, characterized in that The analyzing the target text information by using the pre-trained interruption filtering model to determine whether the voice information belongs to the interruption information includes: The target text information and the historical text information are input into a pre-trained interruption filtering model for interruption analysis to obtain a result of whether the voice information belongs to interruption information; wherein the interruption filtering model is trained based on sample text information, historical sample text information, and the annotation result of whether the sample text information belongs to interruption information; the historical text information represents text information corresponding to historical voice information received within a historical time period before the moment when the voice information is detected.

5. The method according to claim 1, characterized in that The determining the interruption mode corresponding to the voice information includes: The target text information and the historical text information are input into a pre-trained state judgment model to judge the interruption method, and the interruption method corresponding to the voice information is obtained; wherein the state judgment model is trained based on sample text information, historical sample text information, and the marked interruption method corresponding to the sample text information; the historical text information represents the text information corresponding to the historical voice information received within the historical time period before the moment when the voice information is detected.

6. A voice interruption device for an intelligent agent, characterized in that: The device comprises: The voice acquisition module is used to acquire the currently detected voice information when the agent is in a listening state during the process of outputting the response information; A speech recognition module is used to perform speech recognition on the speech information using an automatic speech recognition (ASR) model to obtain target text information; An interruption filtering module, used to analyze the target text information using a pre-trained interruption filtering model to determine whether the voice information is an interruption information; the interruption filtering model is trained based on the BERT model; A mode determination module, used for determining the interruption mode corresponding to the voice information when the voice information belongs to the interruption information; different interruption modes correspond to different states of the intelligent agent; A state switching module, used for switching the agent from a listening state to an interruptible state when the interruption mode corresponding to the voice information is interruptible; A voice interruption module, used for interrupting the intelligent agent from outputting the response information when the intelligent agent pauses in outputting the response information when the intelligent agent is in an interruptible state; The device also includes: The first monitoring module is used to monitor the input voice information when the voice information does not belong to interruption information and there is currently no unoutput response information; or, when the voice information does not belong to interruption information and there is currently no output response information, switch the intelligent agent from the monitoring state to the interruptible state and output the unoutput response information; and when outputting at least one complete response sentence in the unoutput response information, switch the intelligent agent from the interruptible state to the monitoring state to monitor the input voice information.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the voice interruption method for an intelligent agent as described in any one of claims 1-5 when executing the program stored in the memory.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the voice interruption method for an intelligent agent described in any one of claims 1-5 is implemented.

Images