An audio stream cooperative management method and device and a storage medium

Abstract

The application is suitable for the field of artificial intelligence technology, and provides an audio stream cooperative management method, device and storage medium, which comprises the following steps: in an intelligent broadcasting mode, a first audio stream generated by an AI voice agent, a second audio stream collected by a local collection module and / or a third audio stream accessed by a communication module are acquired; the first audio stream is subjected to mirror processing to obtain a mirror audio stream; a software-defined audio matrix is used to mix the mirror audio stream, the second audio stream and / or the third audio stream according to a first weight coefficient to generate a judgment audio stream; a local hardware detection module is used to identify voice energy in the judgment audio stream which is not from the AI voice agent, and whether a user interrupting event exists is judged according to the voice energy; when the user interrupting event exists, the intelligent broadcasting of the AI voice agent is interrupted. The technical scheme of the application can improve the real-time response speed and recognition accuracy of user interrupting detection.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to an audio stream collaborative management method, device and storage medium. Background Technology

[0002] In intelligent voice interaction scenarios, when AI voice agents read text content to users using TTS technology, users often need to interrupt to express new intentions or ask questions. Therefore, how to detect user interruptions in real time and accurately and respond accordingly is a key aspect of improving the naturalness of human-computer interaction and user experience.

[0003] However, existing AI voice agents generally appear stiff in conversational interactions. The fundamental reason for this is that the system cannot detect the user's interruption behavior in real time during the broadcast, that is, it does not have the ability to "stop upon hearing a voice".

[0004] To address these issues, existing technologies often rely on cloud-based voice activity detection solutions. In this approach, the terminal device uploads the audio stream during intelligent playback to a cloud server in real time. The cloud server then performs voice activity detection and semantic understanding on the audio stream to determine if a user has interrupted, and returns the recognition result to the terminal device to trigger a playback interruption. This solution depends on network transmission, and both the uploading of the audio stream and the return of the recognition result require time, resulting in a significant delay in interruption detection. This makes it difficult to meet the need for rapid interruption of user interruptions in real-time interactive scenarios, and fails to achieve a natural "stop upon hearing" interruption experience. Summary of the Invention

[0005] This invention provides an audio stream collaborative management method, which aims to improve the real-time response speed and recognition accuracy of user interruption detection.

[0006] The present invention is implemented as follows: an audio stream collaborative management method includes the following steps: in intelligent broadcast mode, acquiring a first audio stream generated by an AI voice agent, a second audio stream acquired by a local acquisition module, and / or a third audio stream accessed by a communication module; The first audio stream is mirrored to obtain a mirrored audio stream; A software-defined audio matrix is ​​used to mix the mirror audio stream, the second audio stream, and / or the third audio stream according to a first weighting coefficient to generate a decision audio stream; wherein, the weighting coefficient of the mirror audio stream in the first weighting coefficient is less than the weighting coefficient of the second audio stream and / or the third audio stream; The local hardware detection module identifies voice energy in the determined audio stream that does not originate from the AI ​​voice agent, and determines whether a user interruption event exists based on the voice energy. When a user interruption event is detected, the intelligent broadcasting of the AI ​​voice agent is interrupted.

[0007] Furthermore, the local hardware detection module includes a voice activity detection circuit and a level detection circuit. The step of the local hardware detection module identifying voice energy in the determined audio stream that does not originate from the AI ​​voice agent, and determining whether a user interruption event exists based on the voice energy, includes: The speech activity detection circuit compares the acoustic features of the determined audio stream with a preset model to identify speech energy in the determined audio stream that does not originate from the AI ​​speech agent. The energy distribution of the voice energy is detected by the level detection circuit, and it is determined whether the energy distribution is greater than a preset energy threshold. If so, determine that a user interruption occurred; If not, it is determined that there was no user interruption event.

[0008] Furthermore, the local hardware detection module also includes an interrupt controller, and the step of interrupting the intelligent broadcasting of the AI ​​voice agent includes: The interrupt controller generates the highest priority hardware interrupt signal; In response to the highest priority hardware interrupt signal, the interrupt service routine performs minimal atomic operations, records the interrupt flag, and notifies the main control program. The main control program executes an operation to interrupt the intelligent broadcasting of the AI ​​voice agent based on the interrupt flag.

[0009] Furthermore, after the step of interrupting the intelligent broadcasting of the AI ​​voice agent, the method further includes: Control the AI ​​voice agent to stop outputting the first audio stream; The software-defined audio matrix is ​​used to mix the mirror audio stream, the second audio stream, and / or the third audio stream according to a second weighting coefficient to generate a mixed audio stream; wherein, the weighting coefficient of the mirror audio stream in the second weighting coefficient is less than the weighting coefficient of the mirror audio stream in the first weighting coefficient; and the weighting coefficients of the second audio stream and / or the third audio stream in the second weighting coefficient are greater than the weighting coefficients of the second audio stream and / or the third audio stream in the first weighting coefficient; The mixed audio stream is routed to the automatic speech recognition module for user speech recognition.

[0010] Furthermore, after the step of routing the mixed audio stream to the automatic speech recognition module for user speech recognition, the method further includes: The automatic speech recognition module determines whether there is valid user speech in the mixed audio stream; If not, calculate the duration of the absence of the valid user voice; When the duration exceeds the preset duration, the AI ​​voice agent is controlled to switch to low power mode, and the weight coefficients of the software-defined audio matrix are reset to the initial default values.

[0011] Furthermore, after the step of the automatic speech recognition module determining whether there is valid user speech in the mixed audio stream, the method further includes: If present, generate interjection text based on the valid user's speech and send it to the large language model; Receive the response text returned by the large language model based on the interjection text; In the intelligent broadcast mode of the response text instruction, the response text generation mode switching instruction is used; According to the mode switching command, the AI ​​voice agent is controlled to switch back to the intelligent broadcast mode and the first audio stream is resumed.

[0012] The present invention also provides an audio stream collaborative management device, the audio stream collaborative management device including a memory, a processor, and an audio stream collaborative management program stored in the memory and executable on the processor, wherein when the audio stream collaborative management program is executed by the processor, it implements the steps of the audio stream collaborative management method as described in any of the preceding claims.

[0013] The present invention also provides a storage medium storing an audio stream collaborative management program, wherein the audio stream collaborative management program, when executed by a processor, implements the steps of the audio stream collaborative management method described above.

[0014] Because this invention uses a software-defined audio matrix to mix the mirror audio stream of the AI ​​voice agent, the second audio stream acquired by the local acquisition module, and the third audio stream accessed by the communication module according to differentiated weights, it generates a judgment audio stream in which the speech components from the AI ​​sound source are pre-suppressed. The local hardware detection module identifies speech energy in this judgment audio stream that does not originate from the AI ​​voice agent to determine whether a user interruption event exists. When a user interruption event is detected, the intelligent broadcasting of the AI ​​voice agent is immediately interrupted. Therefore, the entire interruption detection and interruption process is completed at the local hardware level, and the detection latency is significantly compressed, truly realizing real-time interruption of AI broadcasting by user speech. At the same time, because the judgment audio stream specifically suppresses speech from the AI ​​sound source and highlights user speech, combined with the dual verification mechanism of the local hardware detection module, the accuracy of interruption detection is greatly improved and false triggers are effectively avoided. In addition, because the software-defined audio matrix supports dynamic adjustment of weight parameters and routing strategies, the system can adapt to the detection requirements of different dialogue stages, significantly improving the naturalness of human-computer interaction and user experience of the AI ​​voice agent. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the first embodiment of the audio stream collaborative management method of the present invention; Figure 2 This is a flowchart illustrating the second embodiment of the audio stream collaborative management method of the present invention; Figure 3 This is a flowchart illustrating the third embodiment of the audio stream collaborative management method of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] The technical solution of this invention acquires a first audio stream generated by an AI voice agent, a second audio stream acquired by a local acquisition module, and / or a third audio stream accessed by a communication module in intelligent broadcast mode. The first audio stream is then mirrored to obtain a mirrored audio stream. Subsequently, a software-defined audio matrix mixes the mirrored audio stream, the second audio stream, and / or the third audio stream according to a first weighting coefficient to generate a judgment audio stream, thus pre-suppressing the speech components from the AI ​​sound source in the judgment audio stream. The judgment audio stream is then analyzed in real time by a local hardware detection module to identify speech energy not emitted by the AI ​​sound source, thereby determining whether a user interruption has occurred. If a user interruption is detected, the intelligent broadcast process of the AI ​​voice agent is directly interrupted. This effectively solves the problems of high detection latency and low recognition accuracy in existing cloud-based voice activity detection schemes.

[0018] Example 1 Reference Figure 1 This invention provides an audio stream collaborative management method, comprising the following steps: Step S1: In intelligent broadcast mode, acquire the first audio stream generated by the AI ​​voice agent, the second audio stream acquired by the local acquisition module, and / or the third audio stream accessed by the communication module. The first audio stream is the downlink broadcast audio signal generated by the TTS module of the AI ​​voice agent. The second audio stream is the audio signal acquired by a local acquisition module (such as a microphone array), including the local user's speech and ambient noise. The third audio stream is the voice signal of the remote caller received through a communication module (such as Bluetooth, VoIP, or a traditional telephone interface). The intelligent broadcast mode represents the dialogue stage where the AI ​​voice agent outputs voice responses to the user through the TTS module.

[0019] Step S2: Mirror the first audio stream to obtain a mirrored audio stream; The system performs real-time copying of the first audio stream output for the AI ​​voice agent to obtain a mirror audio stream. This mirror audio stream is completely consistent with the original first audio stream in terms of audio content, acoustic features, energy distribution, sampling rate, and timing information.

[0020] Step S3: The software defines an audio matrix and mixes the mirror audio stream, the second audio stream, and / or the third audio stream according to a first weighting coefficient to generate a judgment audio stream; wherein, the weighting coefficient of the mirror audio stream in the first weighting coefficient is less than the weighting coefficient of the second audio stream and / or the third audio stream. This software-defined audio matrix is ​​a virtual audio routing and mixing engine running in the terminal operating system or a dedicated processor. It is used to dynamically arrange audio streams, configure weighting coefficients, mix and overlay audio, and route output, without relying on physical audio connections. The first weighting coefficient represents a set of gain weight parameters configured for each audio stream during the mixing process, used to control the energy proportion of each audio stream in the mix. The determined audio stream is the audio signal generated by the software-defined audio matrix after mixing according to the first weighting coefficient. It is used to output to the local hardware detection module for interjection event recognition and is not used for playback to local users or remote clients.

[0021] Specifically, a software-defined audio matrix models, routes, and dynamically mixes the mirror audio stream, the second audio stream, and / or the third audio stream according to a first weighting coefficient, generating a decision audio stream. The weighting coefficient of the mirror audio stream is configured to be less than that of the second and / or third audio streams, thereby suppressing or weakening the speech energy from the AI ​​sound source in the decision audio stream. This enhances the performance of local user speech and / or remote user speech, significantly reducing the difficulty and false positive rate of subsequent local hardware detection modules in detecting user interruptions. Furthermore, the weighting parameters of the software-defined audio matrix can be dynamically adjusted according to the business scenario, exhibiting good flexibility and scenario adaptability.

[0022] Understandably, in scenarios where only local user interruptions need to be detected, the software-defined audio matrix can choose to mix the mirrored audio stream with the second audio stream to obtain the judgment audio stream; in scenarios where remote customer interruptions need to be detected, the software-defined audio matrix can also choose to mix the mirrored audio stream with the third audio stream to obtain the judgment audio stream; or in two-way call scenarios, the three audio streams can be mixed simultaneously to obtain the judgment audio stream, enabling the local hardware detection module to uniformly identify user interruption events from either the local or remote side, and achieving full-scenario coverage of local interruption detection, remote interruption detection, and two-way interruption detection through a single detection architecture.

[0023] Step S4: The local hardware detection module identifies voice energy in the determined audio stream that does not originate from the AI ​​voice agent, and determines whether a user interruption event exists based on the voice energy. The local hardware detection module consists of a hardware voice activity detection circuit (VAD), a level detection circuit, and an interrupt controller. This module is used to identify and determine whether there is third-party voice originating from a non-AI sound source in the audio stream. This third-party voice includes local user voice and / or remote client voice; and based on the acoustic characteristics and energy distribution of the third-party voice, it comprehensively determines whether a user interruption event has occurred.

[0024] Specifically, the local hardware detection module performs real-time analysis of the determined audio stream. Its built-in voice activity detection circuit compares the acoustic features of the determined audio stream with a preset model to identify whether there is user voice in the determined audio stream that differs from the AI ​​mirror audio stream. Then, the level detection circuit analyzes the energy distribution of the non-AI voice energy to determine whether it exceeds a preset energy threshold, thereby determining whether a user interruption event has occurred. By changing the interruption detection from cloud-based voice activity detection to local hardware detection, the detection latency has been significantly reduced from more than 500ms in traditional cloud solutions to less than 10ms, achieving near real-time perception of user interruption behavior.

[0025] In one specific embodiment, step S4 includes: Step S41: The speech activity detection circuit compares the acoustic features of the determined audio stream with a preset model to identify speech energy in the determined audio stream that does not originate from the AI ​​speech agent. Step S42: The level detection circuit detects the energy distribution of the voice energy and determines whether the energy distribution is greater than a preset energy threshold. Step S43: If yes, determine that a user interruption event has occurred; Step S44: If not, determine that there is no user interruption event.

[0026] Specifically, the speech activity detection circuit compares the acoustic features of the determined audio stream with a preset model to identify whether the determined audio stream contains third-party speech energy other than the AI ​​mirrored audio stream, i.e., whether there is speech energy emitted by a non-AI sound source in the determined audio stream. If the acoustic features of the determined audio stream are consistent with the preset model, it indicates that the determined audio stream comes entirely from the AI ​​sound source, and it is determined that there is no user interruption event. If the acoustic features of the determined audio stream deviate from the preset model, it indicates that there is speech energy emitted by a non-AI sound source in the determined audio stream, and it is determined that there may be a user interruption event. Then, the level detection circuit verifies the non-AI speech energy to make a final judgment. The preset model can be a model pre-trained based on the acoustic features of AI speech. The speech activity detection circuit utilizes the inherent differences between AI synthesized speech and user speech in timbre, spectrum, fundamental frequency, etc., to directly distinguish AI speech from non-AI speech at the acoustic feature level, significantly improving the accuracy of interruption detection and reducing the probability of misjudgment caused by environmental noise or fluctuations in the AI ​​speech itself.

[0027] The level detection circuit detects the energy distribution of the voice energy and determines whether the energy distribution is greater than a preset energy threshold. When the level detection circuit determines that the energy distribution of the voice energy emitted by a non-AI sound source is greater than the preset energy threshold, it determines that a user interruption event has occurred. This determination result is output by the level detection circuit to the interrupt controller, triggering the interrupt controller to generate the highest priority interrupt signal and report it to the flow control and state machine module.

[0028] When the level detection circuit determines that the energy distribution of the speech energy emitted by the non-AI sound source in the audio stream is less than the preset energy threshold, it determines that there is no user interruption event. At this time, the interrupt controller does not trigger an interrupt signal, and the system maintains the current AI intelligent broadcasting state.

[0029] In this embodiment, the logic for determining user interruption events includes two levels of verification: The first level involves a voice activity detection circuit comparing the acoustic characteristics of the determined audio stream with a preset model to identify whether there is voice energy not emitted by the AI ​​sound source, thus completing the initial identification of the sound source type. The second level involves a level detection circuit performing amplitude quantization analysis on the non-AI voice energy to determine whether its energy distribution exceeds a preset energy threshold, thus completing the effective verification of signal strength. The first level of verification ensures that the identified voice energy is indeed the user's voice and not from the AI ​​sound source, eliminating the possibility of false detection of the AI-broadcast audio itself. The second level of verification confirms that the loudness level of the user's voice meets the effective interaction standard, eliminating interference from wind, environmental noise, and weak voices from a distance. This significantly improves the accuracy of interruption detection.

[0030] Step S5: When it is determined that the user interruption event exists, the intelligent broadcasting of the AI ​​voice agent is interrupted.

[0031] If a user interruption is detected, the local hardware detection module generates a high-priority hardware interrupt signal. The interrupt operation is then executed by the flow control and state machine module in response to the hardware interrupt signal, immediately interrupting or attenuating the intelligent broadcasting of the AI ​​voice agent. For example, this may involve stopping the AI ​​voice agent's TTS audio output, switching the AI ​​voice agent to listening mode, routing the second audio stream acquired by the local acquisition module or the third audio stream connected to the communication module to the automatic speech recognition (ASR) module for speech recognition, and updating the weight coefficients of the software-defined audio matrix to adapt to the mixing requirements of the listening phase. Through these operations, the AI ​​voice agent can stop broadcasting and switch to listening mode in a very short time after the user interrupts, achieving a natural interactive experience of "stopping upon hearing."

[0032] In one specific embodiment, step S5 includes: Step S51: The interrupt controller generates a hardware interrupt signal with the highest priority. Step S52: In response to the highest priority hardware interrupt signal, the interrupt service routine performs a minimal atomic operation, records the interrupt flag, and notifies the main control program. In step S53, the main control program executes an operation to interrupt the intelligent broadcasting of the AI ​​voice agent based on the interrupt flag.

[0033] The highest priority hardware interrupt signal has the highest response priority among all interrupt sources in the system. It can preempt other low priority tasks that are currently being executed, ensuring that interruption events are processed as soon as possible.

[0034] When the level detection circuit determines that the energy distribution of speech energy not originating from an AI sound source exceeds a preset energy threshold, it outputs a trigger signal to the interrupt controller. Upon receiving the trigger signal, the interrupt controller immediately generates a high-priority hardware interrupt signal and reports it to the flow control and state machine module. In response to the high-priority hardware interrupt signal issued by the interrupt controller, the system's interrupt service routine is immediately triggered. The interrupt service routine is a lightweight processing program that performs only minimal atomic operations to ensure that interrupt processing time is significantly compressed, for example, controlled within 10ms, avoiding blocking the main control program's normal audio stream processing or other real-time tasks.

[0035] After receiving the event notification from the interrupt service routine, the main control program checks the interrupt flag. When it confirms that the interrupt flag is valid, the main control program immediately executes the complete operation of interrupting the AI ​​voice agent intelligent broadcast.

[0036] This invention uses a software-defined audio matrix to mix the mirror audio stream of an AI voice agent, the second audio stream from a local acquisition module, and the third audio stream from a communication module with differentiated weights to generate a judgment audio stream. A local hardware detection module then identifies the speech energy from non-AI sound sources in this judgment audio stream and determines whether a user interruption event exists based on this speech energy. When a user interruption event is detected, the intelligent broadcasting of the AI ​​voice agent is immediately interrupted. The entire interruption detection and interruption process is completed at the local hardware level, significantly reducing detection latency and truly achieving real-time interruption of AI broadcasting by user speech. Simultaneously, by selectively suppressing speech energy from AI sound sources and highlighting user speech energy in the judgment audio stream, combined with the dual verification mechanism of the local hardware detection module, the accuracy of interruption detection is greatly improved and false triggers are effectively avoided. Furthermore, the software-defined audio matrix supports dynamic adjustment of weight parameters and routing strategies, enabling the system to adapt to the detection needs of different dialogue stages, significantly improving the naturalness of human-computer interaction and user experience of the AI ​​voice agent.

[0037] Example 2 Reference Figure 2Based on the above embodiments, after step S5, the method further includes: Step S6: Control the AI ​​voice agent to stop outputting the first audio stream; Step S7: The software defines an audio matrix and mixes the mirror audio stream, the second audio stream, and / or the third audio stream according to a second weighting coefficient to generate a mixed audio stream; wherein, the weighting coefficient of the mirror audio stream in the second weighting coefficient is less than the weighting coefficient of the mirror audio stream in the first weighting coefficient; and the weighting coefficients of the second audio stream and / or the third audio stream in the second weighting coefficient are greater than the weighting coefficients of the second audio stream and / or the third audio stream in the first weighting coefficient; Step S8: The mixed audio stream is routed to the automatic speech recognition module for user speech recognition.

[0038] Upon detecting a user interruption and executing the interrupt intelligent broadcast operation, the flow control and state machine module sends a stop broadcast command to the AI ​​voice processing module, controlling the AI ​​voice agent to immediately stop outputting the first audio stream through the TTS module and enter listening mode. This eliminates interference from the first audio stream on subsequent mixing and speech recognition.

[0039] After the AI ​​broadcast stops outputting, the flow control and state machine module controls the software-defined audio matrix to adjust its weight parameters according to the current state. The software-defined audio matrix remixes the mirror audio stream, the second audio stream, and / or the third audio stream according to the second weight coefficient, generating a mixed audio stream. The key difference between the second and first weight coefficients is that the weight coefficient of the mirror audio stream in the second weight coefficient is smaller than that in the first weight coefficient. This means that the AI ​​mirror audio stream is further suppressed or reduced to zero weight during the listening phase, completely excluding the speech energy from the AI ​​sound source. Furthermore, the weight coefficients of the second and / or third audio streams in the second weight coefficient are greater than those in the first weight coefficient. This means that the user's speech components are further enhanced during the listening phase, maximizing the preservation of the user's voice source energy and significantly improving the accuracy of subsequent speech recognition.

[0040] The mixed audio stream is output to the input of the automatic speech recognition module. After receiving the mixed audio stream, the automatic speech recognition module performs speech recognition processing, converting the user's interjection into text information, which is then parsed by the Large Language Model (LLM) to generate subsequent responses.

[0041] It should be noted that after the AI ​​voice agent stops outputting the first audio stream, the mirrored audio stream, as a copy of the audio stream signal in the software-defined audio matrix, may still exist in the mixing link, interfering with subsequent speech recognition. Therefore, the weight coefficients of the mirrored audio stream are further reduced or reduced to zero to remove residual energy from the AI ​​sound source; simultaneously, the weight coefficients of the second and / or third audio streams are further increased or set to their maximum values ​​to ensure that the automatic speech recognition module receives sufficiently strong user voice input.

[0042] Example 3 Reference Figure 3 Based on the above embodiments, after step S8, the method further includes: Step S9: The automatic speech recognition module determines whether there is valid user speech in the mixed audio stream; Step S10: If not, calculate the duration of the absence of the valid user voice. Step S11: When the duration exceeds the preset duration, control the AI ​​voice agent to switch to low power mode and reset the weight coefficients of the software-defined audio matrix to the initial default value.

[0043] Specifically, the automatic speech recognition module continuously parses and recognizes the mixed audio stream routed by the software-defined audio matrix to determine whether it contains valid user speech. If no valid user speech is recognized, the flow control and state machine module continuously accumulates the duration of no valid user speech. When the duration exceeds the preset duration threshold, it indicates that the user has ended the current round of interruption and the user currently has no intention of continuous interaction. At this time, the flow control and state machine module performs the following operations: on the one hand, it controls the AI ​​voice agent to enter a low-power operation mode to reduce the device's computing power and power consumption; on the other hand, it restores the weight coefficients of the software-defined audio matrix to the initial default parameters, completes the audio mixing path reset, and prepares for re-entering the intelligent broadcast mode.

[0044] Continue to refer to Figure 3 Based on the above embodiments, after step S9, the method further includes: Step S12: If present, generate interjection text based on the valid user speech and send it to the large language model; Step S13: Receive the response text returned by the large language model based on the interjection text; Step S14: When the response text indicates intelligent broadcast mode, switch the response text generation mode based on the response text generation mode switching instruction; Step S15: Control the AI ​​voice agent to switch back to intelligent broadcast mode according to the mode switching instruction, and restore the output of the first audio stream.

[0045] In listening mode, when the automatic speech recognition module detects valid user speech in the mixed audio stream, it performs real-time recognition and text conversion on the collected user speech signal, generating a corresponding interjection text, and transmitting the interjection text to the large language model. Upon receiving the interjection text, the large language model performs semantic analysis of the user's intent, generates a corresponding response text, and feeds it back to the system. After receiving the response text from the large language model, the system determines whether to resume the AI ​​voice agent's intelligent broadcasting based on the control flags carried in the response text. If the control flags indicate that the intelligent broadcasting mode should be enabled, the system generates a mode switching command based on the broadcast content in the response text. In response to the mode switching command, the flow control and state machine module controls the AI ​​voice agent to switch from listening mode back to intelligent broadcasting mode and resumes outputting the first audio stream, thus completing the complete interactive loop from user interjection to AI response.

[0046] The present invention also provides an audio stream collaborative management device, which includes a memory, a processor, and an audio stream collaborative management program stored in the memory and executable on the processor. When the audio stream collaborative management program is executed by the processor, it implements the steps of the audio stream collaborative management method as described in any one of claims 1 to 6.

[0047] The present invention also provides a storage medium storing an audio stream co-management program, which, when executed by a processor, implements the steps of the audio stream co-management method as described in any of the preceding claims.

[0048] It is understood that those skilled in the art can combine various implementation methods in the above embodiments under the guidance of the above examples to obtain technical solutions with multiple implementation methods.

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An audio stream collaborative management method, characterized in that, Includes the following steps: In intelligent broadcast mode, the system acquires the first audio stream generated by the AI ​​voice agent, the second audio stream acquired by the local acquisition module, and / or the third audio stream accessed by the communication module. The first audio stream is mirrored to obtain a mirrored audio stream; A software-defined audio matrix is ​​used to mix the mirror audio stream, the second audio stream, and / or the third audio stream according to a first weighting coefficient to generate a decision audio stream; wherein, the weighting coefficient of the mirror audio stream in the first weighting coefficient is less than the weighting coefficient of the second audio stream and / or the third audio stream; The local hardware detection module identifies voice energy in the determined audio stream that does not originate from the AI ​​voice agent, and determines whether a user interruption event exists based on the voice energy. When a user interruption event is detected, the intelligent broadcasting of the AI ​​voice agent is interrupted.

2. The audio stream collaborative management method as described in claim 1, characterized in that, The local hardware detection module includes a voice activity detection circuit and a level detection circuit. The step of the local hardware detection module identifying voice energy in the audio stream that does not originate from the AI ​​voice agent, and determining whether a user interruption event exists based on the voice energy, includes: The speech activity detection circuit compares the acoustic features of the determined audio stream with a preset model to identify speech energy in the determined audio stream that does not originate from the AI ​​speech agent. The energy distribution of the voice energy is detected by the level detection circuit, and it is determined whether the energy distribution is greater than a preset energy threshold. If so, determine that a user interruption occurred; If not, it is determined that there was no user interruption event.

3. The audio stream collaborative management method as described in claim 2, characterized in that, The local hardware detection module also includes an interrupt controller, and the step of interrupting the intelligent broadcasting of the AI ​​voice agent includes: The interrupt controller generates the highest priority hardware interrupt signal; In response to the highest priority hardware interrupt signal, the interrupt service routine performs a minimal atomic operation, records the interrupt flag, and notifies the main control program. The main control program executes an operation to interrupt the intelligent broadcasting of the AI ​​voice agent based on the interrupt flag.

4. The audio stream collaborative management method as described in claim 1, characterized in that, After the step of interrupting the intelligent broadcasting of the AI ​​voice agent, the method further includes: Control the AI ​​voice agent to stop outputting the first audio stream; The software-defined audio matrix is ​​used to mix the mirror audio stream, the second audio stream, and / or the third audio stream according to a second weighting coefficient to generate a mixed audio stream; wherein, the weighting coefficient of the mirror audio stream in the second weighting coefficient is less than the weighting coefficient of the mirror audio stream in the first weighting coefficient; and the weighting coefficients of the second audio stream and / or the third audio stream in the second weighting coefficient are greater than the weighting coefficients of the second audio stream and / or the third audio stream in the first weighting coefficient; The mixed audio stream is routed to the automatic speech recognition module for user speech recognition.

5. The audio stream collaborative management method as described in claim 4, characterized in that, Following the step of routing the mixed audio stream to the automatic speech recognition module for user speech recognition, the method further includes: The automatic speech recognition module determines whether there is valid user speech in the mixed audio stream; If not, calculate the duration of the absence of the valid user voice; When the duration exceeds the preset duration, the AI ​​voice agent is controlled to switch to low power mode, and the weight coefficients of the software-defined audio matrix are reset to the initial default values.

6. The audio stream collaborative management method as described in claim 5, characterized in that, After the step of determining whether there is valid user speech in the mixed audio stream by the automatic speech recognition module, the method further includes: If present, generate interjection text based on the valid user's speech and send it to the large language model; Receive the response text returned by the large language model based on the interjection text; In the intelligent broadcast mode of the response text instruction, the response text generation mode switching instruction is used; According to the mode switching command, the AI ​​voice agent is controlled to switch back to the intelligent broadcast mode and the first audio stream is resumed.

7. An audio stream collaborative management device, characterized in that, The audio stream collaborative management device includes a memory, a processor, and an audio stream collaborative management program stored in the memory and executable on the processor. When the audio stream collaborative management program is executed by the processor, it implements the steps of the audio stream collaborative management method as described in any one of claims 1 to 6.

8. A storage medium, characterized in that, The storage medium stores an audio stream collaborative management program, which, when executed by a processor, implements the steps of the audio stream collaborative management method as described in any one of claims 1 to 6.

Images