Voice activity signal detection method and voice activity signal detection system

Abstract

The invention discloses a voice activity signal detection method and a voice activity signal detection system, and relates to the field of audio signal processing, and the method comprises the steps: a neural network voice activity detection module receives at least one path of oversampled initial audio sequence collected by a signal collection module; sequentially performing data preprocessing, feature vector coding processing, vector fusion processing, voice activity recognition processing and signal conversion processing based on the oversampled initial audio sequence through a neural network voice activity detection model to obtain a voice activity signal; the power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module and generates a feedback control signal to control the working mode of the signal acquisition module; the working mode comprises a low power consumption mode and a normal mode. Detection time delay can be reduced, and the real-time performance of subsequent voice interaction can be improved; and the overall power consumption of the system is reduced.

Description

Technical Field

[0001] The present application relates to the field of audio signal processing, and in particular to a voice activity signal detection method and a voice activity signal detection system. Background Art

[0002] Speech is the most fundamental and natural form of human-computer interaction. Efficient and accurate speech processing is a crucial technical step in voice interaction. Voice activity detection (VAD) is a crucial step before speech processing. After identifying valid speech segments and invalid noise segments through VAD, the identified segments are then subjected to subsequent speech processing. Common speech processing methods include speech coding, speech noise reduction, speech recognition, and speech enhancement. Currently, the mainstream VAD technology solution uses a microphone array to capture sound signals. A multi-channel ADC (analog-to-digital converter) converts the captured sound signals into multiple channels of audio data, each consisting of valid speech segments and invalid noise segments. These multiple channels of audio data are transmitted via a dedicated data interface to a general-purpose computing platform. The VAD algorithm is first run on the platform to process the multiple channels of audio data and generate VAD signals. Subsequent speech algorithms then perform more complex speech processing based on the VAD signals. This requires the entire system to be constantly operational to respond to voice commands, resulting in high overall system power consumption. In addition, the sampling frequency of raw voice data is generally low, usually 16kHz. When a general computing platform adopts a slightly more complex VAD algorithm, it requires more time cycles to calculate, which will lead to increased data processing delays and poor real-time voice interaction. Summary of the Invention

[0003] This application aims to solve at least one of the technical problems existing in the prior art. To this end, this application proposes a voice activity signal detection method and voice activity signal detection system. These methods can detect an oversampled initial audio sequence to obtain a voice activity signal, thereby reducing detection latency and improving the real-time performance of subsequent voice interactions. Furthermore, they use the voice activity signal to perform power consumption feedback control, thereby reducing overall system power consumption.

[0004] In a first aspect, embodiments of the present application provide a voice activity signal detection method, which is applied to a voice activity signal detection system. The system comprises: a neural network voice activity detection module, a signal acquisition module and a power consumption feedback control module communicatively connected to the neural network voice activity detection module; the power consumption feedback control module is further communicatively connected to the signal acquisition module; wherein the neural network voice activity detection module comprises: a pre-trained neural network voice activity detection model; The method comprises: The neural network voice activity detection module receives at least one oversampled initial audio sequence collected by the signal acquisition module; The neural network voice activity detection model sequentially performs data preprocessing, feature vector encoding, vector fusion, voice activity recognition, and signal conversion based on the oversampled initial audio sequence to obtain a voice activity signal. The voice activity recognition process is used to decode the output multi-head attention vector at the current moment to obtain a voice activity probability sequence. The signal conversion process is used to perform threshold judgment on the voice activity probability sequence, convert the voice activity probability sequence into a voice activity signal based on the obtained judgment result, and output the voice activity signal. The power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module and generates a feedback control signal to control the working mode of the signal acquisition module; the working mode includes: low power consumption mode and normal mode.

[0005] In a second aspect, the present application provides a voice activity signal detection system, comprising: a neural network voice activity detection module, a signal acquisition module and a power consumption feedback control module communicatively connected to the neural network voice activity detection module; the power consumption feedback control module is also communicatively connected to the signal acquisition module; wherein the neural network voice activity detection module comprises: a pre-trained neural network voice activity detection model; the signal acquisition module, the neural network voice activity detection module, and the power consumption feedback control module cooperate with each other to enable the voice activity signal detection system to implement the voice activity signal detection method as described in any one of the embodiments of the first aspect.

[0006] The embodiment of the present application includes: a voice activity signal detection system including: a neural network voice activity detection module, a signal acquisition module and a power consumption feedback control module in communication with the neural network voice activity detection module; the power consumption feedback control module is also in communication with the signal acquisition module; wherein the neural network voice activity detection module includes: a pre-trained neural network voice activity detection model; in the process of performing voice activity signal detection processing using the voice activity signal detection system, first, the neural network voice activity detection module receives at least one oversampled initial audio sequence acquired by the signal acquisition module; directly acquires the oversampled initial audio sequence to lay a data foundation for subsequent relatively rapid voice activity detection; then, through the neural network voice activity detection model, data preprocessing, feature vector processing are sequentially performed based on the oversampled initial audio sequence. The system undergoes a series of processing steps, including encoding, vector fusion, voice activity recognition, and signal conversion, to obtain a voice activity signal. The voice activity recognition process is used to decode the multi-head attention vector output at the current moment to obtain a voice activity probability sequence. The signal conversion process is used to perform threshold judgment on the voice activity probability sequence, convert the voice activity probability sequence into a voice activity signal based on the judgment result, and output the voice activity signal. By directly performing voice activity detection on the oversampled initial audio sequence using a neural network voice activity detection model, the detection delay is reduced, which is beneficial for improving the real-time performance of subsequent voice interaction. Finally, the power consumption feedback control module monitors the level of the voice activity signal input by the neural network voice activity detection module and generates a feedback control signal to control the operating mode of the signal acquisition module. The operating modes include low power consumption mode and normal mode. Power consumption feedback control is performed using the obtained voice activity signal to dynamically adjust the operating mode of the signal acquisition module, thereby effectively reducing the overall power consumption of the system. That is to say, the solution of the embodiment of the present application can detect the oversampled initial audio sequence to obtain a voice activity signal, and perform power consumption feedback control through the voice activity signal, thereby reducing the detection delay, which is conducive to improving the real-time performance of subsequent voice interaction; and effectively reducing the overall power consumption of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 1 is a schematic diagram of the system architecture of a voice activity signal detection system for executing a voice activity signal detection method provided by one embodiment of the present application; Figure 2 is a schematic diagram of a traditional voice activity signal detection architecture provided by one embodiment of the present application; Figure 3 This is a specific schematic diagram of a voice activity signal detection system provided by an embodiment of the present application; Figure 4This is a schematic diagram of the specific structure of an analog-to-digital converter provided by an embodiment of the present application; Figure 5 This is a schematic diagram of the specific structure of a neural network voice activity detection model provided by one embodiment of the present application; Figure 6 This is a schematic diagram of the specific structure of a voice activity signal detection system implemented based on an ASIC chip provided by one embodiment of the present application; Figure 7 This is a flow chart of a voice activity signal detection method provided by one embodiment of the present application; Figure 8 This is a flowchart of a power consumption feedback control process provided by an embodiment of the present application; Figure 9 This is a timing diagram of a voice activity signal and a feedback control signal provided by an embodiment of the present application; Figure 10 This is a flowchart of training a neural network voice activity detection model provided by an embodiment of the present application. DETAILED DESCRIPTION

[0008] In order to make the purpose, technical solutions and advantages of this application more clear, this application is further described in detail below with reference to the accompanying drawings and embodiments.

[0009] It should be noted that although a logical order is shown in the flowchart in the description of this application, in some cases, the steps shown or described may be performed in an order different from that in the flowchart. In the description of this application, "several" means one or more, and "more" means two or more. The description of "first" and "second" is only used to distinguish technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0010] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0011] First, some terms used in this application are explained: Voice Activity Detection (VAD) is a widely used technology in speech processing. After identifying valid speech segments and invalid noise segments through VAD, subsequent speech processing is performed on the identified segments. Common speech processing techniques include speech coding, speech recognition, speech noise reduction, and speech enhancement. In speech coding, frames with speech activity must be identified and encoded, saving approximately 50% of transmission bandwidth. In speech recognition, segments containing human voices must be extracted for processing by the speech recognition algorithm, conserving system computing resources. In speech noise reduction / enhancement, non-speech segments must be removed or enhanced. While VAD focuses on different application scenarios, its essence is to detect and distinguish between speech and non-speech segments. Accuracy and latency are two important evaluation criteria, and they even directly impact the human-computer interaction experience. Among them, the accuracy rate is evaluated from two dimensions: false detection rate and missed detection rate. The false detection rate refers to the ratio of non-speech segments being detected as speech segments, and the missed detection rate refers to the ratio of speech segments being missed as non-speech segments. A high accuracy rate means low false detection rate and missed detection rate.

[0012] The present application provides a voice activity signal detection method and system, relating to the field of audio signal processing. The method comprises: a neural network voice activity detection module receives at least one oversampled initial audio sequence acquired by a signal acquisition module; a neural network voice activity detection model sequentially performs data preprocessing, feature vector encoding, vector fusion, voice activity recognition, and signal conversion based on the oversampled initial audio sequence to obtain a voice activity signal; a power consumption feedback control module monitors the level of the voice activity signal transmitted by the neural network voice activity detection module and generates a feedback control signal to control the operating mode of the signal acquisition module; the operating modes include low power consumption mode and normal mode. This can reduce detection delay, thereby improving the real-time performance of subsequent voice interactions and reducing overall system power consumption.

[0013] The embodiments of the present application are further described below with reference to the accompanying drawings.

[0014] like Figure 1As shown, the voice activity signal detection system 100 includes: a neural network voice activity detection module 120, a signal acquisition module 110 and a power consumption feedback control module 130 in communication with the neural network voice activity detection module 120; the power consumption feedback control module 130 is also in communication with the signal acquisition module 110; wherein the neural network voice activity detection module 120 includes: a pre-trained neural network voice activity detection model; the signal acquisition module 110, the neural network voice activity detection module 120, and the power consumption feedback control module 130 cooperate with each other to enable the voice activity signal detection system to implement the voice activity signal detection method of the embodiment of the present application. The voice activity signal detection system provided in the embodiment of the present application can detect an oversampled initial audio sequence to obtain a voice activity signal, thereby reducing detection delay and facilitating the real-time performance of subsequent voice interaction; and power consumption feedback control through the voice activity signal is conducive to reducing the overall power consumption of the system.

[0015] Specifically, in the voice activity signal detection system 100, the neural network voice activity detection module 120 receives at least one oversampled initial audio sequence from the signal acquisition module 110. Using the neural network voice activity detection model, the oversampled initial audio sequence undergoes data preprocessing, feature vector encoding, vector fusion, voice activity recognition, and signal conversion to generate a voice activity signal. The power consumption feedback control module 130 monitors the level of the voice activity signal received from the neural network voice activity detection module 120 and generates a feedback control signal to control the operating mode of the signal acquisition module 110. The operating modes include low power mode and normal mode. Power consumption feedback control is performed using the obtained voice activity signal to dynamically adjust the operating mode of the signal acquisition module, thereby effectively reducing overall system power consumption.

[0016] It is understandable that traditional voice activity detection architectures such as Figure 2As shown in the figure, the traditional voice activity detection architecture includes one or more signal conversion pathways, a downsampling module, a general-purpose computing platform, and memory. The signal conversion pathway includes a microphone and an analog-to-digital converter (ADC). Specifically, the microphone, ADC, general-purpose computing platform, and memory are sequentially connected. The most commonly used voice activity detection method in the industry currently uses one or more signal conversion pathways (including microphones and ADCs) to collect oversampled audio data. The oversampled audio data is first decimated by a downsampling module to obtain standard audio data, typically at a rate of 16kHz. This standard audio data is then sent to a general-purpose computing platform, where a complex VAD algorithm is executed to generate a VAD signal. The general-purpose computing platform and associated storage can be independent of the voice front-end in any form factor, including a CPU, NPU, GPU, FPGA, and ASIC, or integrated with the voice front-end. Traditional voice activity detection methods require the entire detection system to be constantly operational to respond to voice commands, resulting in high overall system power consumption. Moreover, the sampling frequency of the original voice data processed by the detection system is generally low, usually 16kHz. When a general computing platform adopts a slightly more complex VAD algorithm, it requires more time cycles to calculate, which will lead to increased data processing delays and poor real-time voice interaction. It can be seen that the disadvantages of using the traditional voice activity detection architecture for voice activity detection are: high power consumption and large delays. However, after the voice activity signal detection system provided in the embodiment of the present application collects the oversampled initial audio sequence, it does not first use the standard sampling rate to downsample the oversampled initial audio sequence to be processed. Instead, it uses a neural network voice activity detection module to directly process the collected oversampled initial audio sequence and output a voice activity signal, eliminating the downsampling process in the ADC conversion process. The signal acquisition and processing speed is faster, and the detection delay of the overall voice activity detection process is reduced, which is conducive to improving the real-time performance of subsequent voice interaction.

[0017] Further, combined with Figure 3 The signal acquisition module 110 is further described. According to some embodiments of the present application, the signal acquisition module 110 includes: at least one audio acquisition unit 111; each audio acquisition unit 111 is used to acquire an initial oversampled audio sequence; and enters a low power mode or a normal mode under the control of the power consumption feedback control module 130.

[0018] In some embodiments, each audio acquisition unit 111 includes a microphone 111A and an analog-to-digital converter (ADC). The voice activity signal detection system 100 further includes a signal input terminal 101. Microphone 111A is communicatively connected to signal input terminal 101 and is configured to receive an analog signal inputted through signal input terminal 101 and input the analog signal into the analog-to-digital converter (ADC). The ADC performs analog-to-digital conversion on the input analog signal to obtain an oversampled initial audio sequence. The oversampled initial audio sequence is then inputted into the neural network voice activity detection module 120.

[0019] Specifically, combined Figure 4 , further describing the analog-to-digital converter ADC. In some embodiments, Figure 4 As shown, the analog-to-digital converter (ADC) includes a programmable gain amplifier 1121 and a Sigma-Delta modulator 1122, which are communicatively connected in sequence. The input of the programmable gain amplifier 1121 is connected to the output of the microphone 111A, and the output of the Sigma-Delta modulator 1122 is connected to the input of the neural network voice activity detection module 120. Specifically, the programmable gain amplifier 1121 is configured to receive the analog signal (voltage signal) output by the microphone 111A and output the amplified analog signal to the Sigma-Delta modulator 1122. The Sigma-Delta modulator 1122 modulates the amplified analog signal into an oversampled initial audio sequence (i.e., a digital signal), and inputs the oversampled initial audio sequence into the neural network voice activity detection module 120.

[0020] Specifically, the programmable gain amplifier 1121 can amplify the analog signal (voltage signal) according to a given gain. If the system requires a gain of 1, the programmable gain amplifier can be omitted. The Sigma-Delta modulator 1122 converts the amplified voltage signal into oversampled audio data at a fixed rate. The conversion rate = standard audio rate * oversampling rate (OSR). For most speech recognition scenarios, the standard audio rate is 16kHz. The oversampling rate is design-related, and common values are 64, 128, and 256.

[0021] In the embodiment of the present application, the microphone 111A and the analog-to-digital converter ADC in the signal acquisition module 110 cooperate with each other to directly acquire an oversampled initial audio sequence, thereby laying a data foundation for subsequent relatively fast voice activity detection.

[0022] In another embodiment, the audio acquisition unit 111 may only include: a digital microphone for outputting digital signals. Specifically, the microphone for outputting digital signals may be implemented using a silicon microphone.

[0023] It can be understood that the microphone and the analog-to-digital converter work together to output the collected sound signal as a digital signal, and the digital microphone can output the collected sound signal as a digital signal; it can be seen that when the audio acquisition unit adopts a microphone and an analog-to-digital converter, or only a digital microphone, the same function can be achieved; therefore, this application does not impose specific restrictions on the specific structural composition and implementation method of the audio acquisition unit.

[0024] The neural network voice activity detection module 120 is further described. The neural network voice activity detection module 120 includes: a pre-trained neural network voice activity detection model; specifically, the network voice activity detection model is based on a multi-head self-attention mechanism.

[0025] like Figure 5 As shown, according to some embodiments of the present application, the neural network voice activity detection module 120 includes: a neural network voice activity detection model 121, and the neural network voice activity detection model 121 includes: a speech input layer, a feature vector encoding layer, a feature integration layer, a feature vector decoding and recognition layer, and an output layer connected in sequence.

[0026] Among them, the speech input layer is used to: perform data preprocessing on the oversampled initial audio sequence to obtain a corresponding input matrix; wherein the input matrix includes multiple elements, and different elements represent the feature vectors corresponding to different frame moments; the feature vector encoding layer is used to: perform feature vector encoding processing according to the input matrix to obtain at least one feature vector; the feature integration layer is used to: perform vector fusion processing on at least one feature vector according to the pre-obtained first weight matrix and the preset integration formula to integrate and obtain the multi-head attention vector at the current moment; the feature vector decoding and recognition layer is used to: perform speech activity recognition processing according to the multi-head attention vector at the current moment to obtain a speech activity probability sequence; the output layer is used to perform threshold judgment on the speech activity probability sequence, and based on the obtained judgment result, convert the speech activity probability sequence into a speech activity signal, and output the speech activity signal.

[0027] Specifically, if Figure 5As shown, the speech input layer includes multiple speech input units, which are responsible for receiving multiple speech signals. Each speech signal is represented as a separate sequence. Since the self-attention mechanism is insensitive to position information, position encoding is introduced in the speech input layer so that the neural network speech activity detection model can understand the timing information in the input initial audio sequence in subsequent processing. In addition, the speech input layer is responsible for preprocessing the input oversampled initial audio sequence to obtain an input matrix. This data preprocessing includes but is not limited to: frame processing, original feature extraction processing, dimensionality expansion processing, and position encoding processing. The input matrix includes multiple elements, and different elements represent the feature vectors corresponding to different frame moments; that is, the input matrix includes the basic features and timing information of the elements.

[0028] Specifically, if Figure 5 As shown in the figure, the feature vector encoding layer includes multiple transformer encoders, each connected to a corresponding speech input unit, receiving an input matrix. In the feature vector encoding layer, each initial audio sequence is encoded into a high-dimensional feature vector based on a multi-head self-attention mechanism, providing the data foundation for the subsequent integrated multi-head attention vector.

[0029] Specifically, the feature integration layer integrates the multi-head self-attention mechanism and uses the pre-obtained first weight matrix to perform weighted fusion on the high-dimensional feature vectors output by each encoder to obtain the target integration vector (multi-head attention vector) at the current moment, thereby extracting and understanding global context information.

[0030] Specifically, if Figure 5 As shown, the feature vector decoding layer includes a lightweight temporal classifier that combines the multi-head attention vectors of the previous moment and the current moment to decode and generate a speech activity probability sequence.

[0031] Specifically, combined Figure 3 and Figure 5 The output layer converts the voice activity probability sequence into a voice activity signal corresponding to the audio data. The voice activity signal is output to the power consumption feedback control module and to an external device via the first output terminal 102 of the voice activity signal detection system. In another embodiment, the output layer can also be configured to output oversampled initial audio data. Alternatively, when the neural network voice activity detection model 121 also integrates a noise reduction function, the output layer can also be configured to output oversampled audio data after noise reduction. This will not be described in detail.

[0032] It can be understood that through the mutual cooperation of the voice input layer, feature vector encoding layer, feature integration layer, feature vector decoding and recognition layer and output layer, the collected oversampled initial audio sequence can be directly subjected to voice activity detection processing, and the voice activity signal can be output quickly, thereby reducing the detection delay and improving the real-time performance of subsequent voice interaction.

[0033] According to some embodiments of the present application, the neural network voice activity detection module is a hardware module implemented based on FPGA or ASIC.

[0034] It's understandable that the neural network voice activity detection model is a pre-trained Transformer network. Pre-training can fix most parameters, allowing it to be implemented using pure logic circuits. Unlike software-based voice activity detection methods, the neural network voice activity detection module in the embodiments of this application is hardware-based and does not require general-purpose computing modules such as a CPU or DSP, nor does it require memory. This reduces hardware costs.

[0035] According to some embodiments of the present application, the voice activity signal detection system further includes: a parameter adjustment module, which is electrically connected to the neural network voice activity detection module; the parameter adjustment module is used to adjust parameters in the hardware neural network voice activity detection module.

[0036] It is understandable that in the process of pre-training the neural network model, the noise reduction model is generally trained using a large amount of multi-channel voice data in various environments. Therefore, a large number of parameters of the noise reduction model obtained after training are fixed, so that the neural network voice activity detection module including the neural network voice activity detection model can be hardware-based using FPGA or ASIC. It should be noted that in order to make the hardware-based neural network voice activity detection module have better adaptability and better suited to different specific scenarios, it is also possible to selectively retain some parameters in the neural network voice activity detection model for fine-tuning.

[0037] In some embodiments, the downsampling filter module is used to convert low-precision oversampled data into high-precision standard sampling rate data, which is generally achieved by a combination of FIR filters and IIR filters. The current traditional speech recognition method generally converts the standard sampling rate in the cloud, and then performs speech recognition based on the conversion to the standard sampling rate, which has ensured the accuracy and timeliness of speech recognition. The embodiment of the present application sets a downsampling filter module to downsample the oversampled data output by the neural network voice activity detection model to adapt to the subsequent speech recognition and meet the accuracy and timeliness of the subsequent speech recognition. And there is no need to upload multiple channels of voice data to the cloud, which greatly reduces the requirements for network bandwidth and hardware.

[0038] In some embodiments, as Figure 3 As shown, the voice activity signal detection system 100 further includes a downsampling filter module 140, a first output terminal 102, and a second output terminal 103. The input terminal of the downsampling filter module 140 is connected to another output terminal of the neural network voice activity detection module 120, and the output terminal of the downsampling filter module 140 is connected to the second output terminal 103. The second output terminal 103 is configured to output the downsampled original audio data simultaneously with the output of the voice activity signal at the first output terminal 102. Alternatively, when the neural network voice activity detection module 120 also integrates a noise reduction function, the second output terminal 103 is configured to output the downsampled audio data after noise reduction simultaneously with the output of the voice activity signal at the first output terminal 102.

[0039] The downsampling filter module 140 is further explained. The function of the downsampling filter module 140 is to extract low-precision oversampled audio data into high-precision standard sampling rate audio data, which is generally achieved by using an FIR filter or an IIR filter. At this stage, most speech processing is based on audio data at a standard rate, such as 16KHz. The current traditional speech recognition method generally converts the standard sampling rate in the cloud, and then performs speech recognition based on the conversion to the standard sampling rate to ensure the accuracy and timeliness of speech recognition. The embodiment of the present application sets a downsampling filter module to downsample the oversampled data output by the neural network speech activity detection model to adapt to the subsequent speech recognition and meet the accuracy and timeliness of the subsequent speech recognition. And there is no need to upload multiple channels of voice data to the cloud, which greatly reduces the requirements for network bandwidth and hardware.

[0040] For example, Figure 6As shown, the voice activity signal detection system is fully implemented as an ASIC chip. This ASIC integrates two analog-to-digital converters (ADCs), each used to convert analog electrical signals from analog microphones MIC0 and MIC1 into digital signals. It also directly receives PDM (Pulse Density Modulation) signals from digital microphones DMIC#0 and DMIC#1. A first multiplexer (mux0), connected to ADC0 and digital microphone DMIC#0, selects one data path from the converted digital and PDM signals and outputs it to a pre-trained neural network voice activity detection module. A second multiplexer (mux1), connected to ADC1 and digital microphone DMIC#1, selects one data path from the converted digital and PDM signals and outputs it to the pre-trained neural network voice activity detection module. After processing by the neural network voice activity detection module, a voice activity signal and oversampled data are generated. This oversampled data is then converted to a standard sampling rate through a downsampling filter module. This standard sampling rate data is then output to other models via a standard audio IIS interface for subsequent speech recognition, semantic understanding, and other processing. As you can understand, the ASIC chip also reserves some adjustable parameters for the neural network voice activity detection module to meet specific application needs. Parameter adjustment is achieved through the parameter adjustment module and the IIC interface. Furthermore, the ASIC chip also integrates the necessary system modules such as the power supply, reference voltage VREF, clock signal, and power consumption feedback control.

[0041] In summary, in the voice activity signal detection system provided in the embodiment of the present application, the VAD function is separated from the general computing platform, and the timing of implementing the VAD function is advanced to after the ADC output from the system architecture. A set of pre-trained neural network voice activity detection models based on the multi-head self-attention mechanism are run to directly process the oversampled audio data and output the VAD signal (i.e., voice activity signal), thereby realizing the VAD function; achieving the advantages of high detection accuracy, low power consumption, low latency, and strong versatility.

[0042] Those skilled in the art will understand that the system structure shown in the figure does not constitute a limitation on the embodiments of the present application, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0043] Those skilled in the art will understand that the system architecture and application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will know that with the evolution of the system architecture and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0044] Based on the above system structure, various embodiments of the voice activity signal detection method of the present application are proposed below.

[0045] like Figure 7 As shown, the voice activity signal detection method is applied to Figure 1 In the voice activity signal detection system shown, the voice activity signal detection system includes: a neural network voice activity detection module, a signal acquisition module and a power consumption feedback control module that are communicatively connected to the neural network voice activity detection module; the power consumption feedback control module is also communicatively connected to the signal acquisition module; wherein the neural network voice activity detection module includes: a pre-trained neural network voice activity detection model; the voice activity signal detection method may include but is not limited to steps S110 to S130.

[0046] Step S110: The neural network voice activity detection module receives at least one oversampled initial audio sequence collected by the signal collection module.

[0047] Step S120: Using a neural network voice activity detection model, data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing, and signal conversion processing are sequentially performed based on the oversampled initial audio sequence to obtain a voice activity signal; wherein, the voice activity recognition processing is used to decode the multi-head attention vector output at the current moment to obtain a voice activity probability sequence; the signal conversion processing is used to perform threshold judgment on the voice activity probability sequence, and based on the obtained judgment result, the voice activity probability sequence is converted into a voice activity signal, and the voice activity signal is output.

[0048] Step S130: The power consumption feedback control module monitors the level of the voice activity signal transmitted by the neural network voice activity detection module and generates a feedback control signal to control the working mode of the signal acquisition module; the working modes include: low power consumption mode and normal mode.

[0049] It can be understood that each oversampled initial audio sequence is a separate input sequence, wherein the initial audio sequence includes multiple elements, and each element is a multi-dimensional vector.

[0050] It can be understood that the level states of the voice activity signal include: high level and low level.

[0051] It can be understood that the feedback control signal includes a first control signal (high level, value 1) and a second control signal (low level, value 0).

[0052] It can be understood that through steps S110 and S120, the oversampled initial audio sequence is directly processed, and the downsampling process in the ADC conversion process is omitted in step S110, which makes the signal acquisition speed faster and the overall voice activity detection processing delay shorter.

[0053] The data source processed by the embodiment of the present application is the high-speed audio signal before extraction (the oversampled initial audio sequence), which naturally has a faster processing speed and thus lower latency. In addition, the final oversampled initial audio sequence must be extracted before being sent to the general computing platform, which is equivalent to the voice activity signal being "pre-processed". This will greatly reduce or even eliminate the need for data buffers, and the overall hardware overhead is smaller.

[0054] Through steps S110 to S130, in the process of performing voice activity signal detection processing using the voice activity signal detection system, first, the neural network voice activity detection module receives at least one oversampled initial audio sequence collected by the signal acquisition module; directly collects the oversampled initial audio sequence to lay a data foundation for subsequent relatively rapid voice activity detection; then, through the neural network voice activity detection model, based on the oversampled initial audio sequence, data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing and signal conversion processing are sequentially performed to obtain a voice activity signal; wherein, the voice activity recognition processing is used to output the current moment according to the The multi-head attention vector is decoded to obtain a speech activity probability sequence; the signal conversion processing is used to perform threshold judgment on the speech activity probability sequence, and based on the obtained judgment result, the speech activity probability sequence is converted into a speech activity signal, and the speech activity signal is output; by directly performing speech activity detection on the oversampled initial audio sequence using the neural network speech activity detection model, the detection delay is reduced, which is conducive to improving the real-time performance of subsequent voice interaction; finally, the power consumption feedback control module monitors the level state of the voice activity signal input by the neural network speech activity detection module and generates a feedback control signal to control the working mode of the signal acquisition module; the working modes include: low power mode and normal mode. Power consumption feedback control is performed through the obtained voice activity signal, and the working mode of the signal acquisition module is dynamically adjusted, thereby effectively reducing the overall power consumption of the system. That is, the solution of the embodiment of the present application can detect the oversampled initial audio sequence to obtain a speech activity signal, thereby reducing the detection delay, which is conducive to improving the real-time performance of subsequent voice interaction; and power consumption feedback control is performed through the voice activity signal, effectively reducing the overall power consumption of the system.

[0055] According to some embodiments of the present application, a neural network voice activity detection model includes: a voice input layer, a feature vector encoding layer, a feature integration layer, a feature vector decoding and recognition layer, and an output layer connected in sequence; further explaining step S120, step S120: through the neural network voice activity detection model, based on the oversampled initial audio sequence, data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing, and signal conversion processing are performed in sequence to obtain a voice activity signal, including but not limited to steps S121 to S125.

[0056] Step S121: Preprocess the oversampled initial audio sequence through the speech input layer to obtain a corresponding input matrix; wherein the input matrix includes multiple elements, and different elements represent feature vectors corresponding to different frame times.

[0057] Step S122: performing feature vector encoding processing according to the input matrix through the feature vector encoding layer to obtain at least one feature vector.

[0058] Step S123: Through the feature integration layer, vector fusion processing is performed on at least one feature vector according to the pre-obtained first weight matrix and the preset integration formula to integrate and obtain the multi-head attention vector at the current moment.

[0059] Step S124: Through the feature vector decoding recognition layer, voice activity recognition processing is performed according to the multi-head attention vector at the current moment to obtain a voice activity probability sequence.

[0060] Step S125: performing threshold judgment on the voice activity probability sequence through the output layer, converting the voice activity probability sequence into a voice activity signal based on the obtained judgment result, and outputting the voice activity signal.

[0061] Further describing step S121, data preprocessing is performed on the input oversampled initial audio sequence. This data preprocessing includes, but is not limited to, framing, original feature extraction, dimensionality expansion, and position encoding. The input matrix obtained after data preprocessing includes multiple elements, each representing a feature vector corresponding to a different frame time; that is, the input matrix includes the basic features and timing information of the elements.

[0062] Specifically, the input oversampled initial audio sequence is preprocessed to obtain the input matrix X, which can be expressed as: ; where each element of the input matrix X It is a d-dimensional vector, i represents different frame moments, d represents the feature dimension, and T represents the T-th frame, that is, the number of frames.

[0063] Further explanation of step S122, wherein eigenvector encoding processing is performed according to the input matrix to obtain at least one eigenvector, includes but is not limited to steps S1221 to S1224.

[0064] Step S1221: According to the preset weight matrix obtained by pre-training, the input matrix is linearly transformed to obtain a projection matrix; the projection matrix includes: query matrix Q, key matrix K and value matrix V. Specifically, ;in, is the second weight matrix obtained after training, is the third weight matrix obtained through training, is the fourth weight matrix obtained through training. Specifically, the second weight matrix, the third weight matrix, and the fourth weight matrix all come from the same characteristic matrix X of the previous layer, that is, the input initial audio sequence is linearly mapped multiple times to obtain different projection matrices.

[0065] Step S1222: Calculate the dot product of the query matrix Q and the key matrix K, and divide the dot product by the dimension of the key matrix K , we get the attention score matrix. The calculation expression of the attention score matrix is: .

[0066] Step S1223: Calculate the attention output matrix using the activation parameters to obtain the attention weight matrix. That is, apply the activation function (i.e., Softmax function) to each row of the attention score matrix to obtain the attention weight matrix. The calculation expression of the attention weight matrix is: .

[0067] Step S1224: Perform weighted summation on the value matrix V according to the attention weight matrix to obtain the eigenvector; the eigenvector is the output matrix. The expression of the output matrix is: .

[0068] Through steps S1221 to S1224, through the multi-head self-attention mechanism, multiple groups of linear transformations are added on the basis of the self-attention mechanism, so that the neural network voice activity detection model can focus on different positions in different subspaces in parallel, thereby capturing more information and achieving better detection effects.

[0069] Further explaining step S123, specifically, step S123 is used to splice the feature vectors output under the multi-head attention mechanism, and obtain the multi-head attention vector after feature fusion after linear transformation. The preset integration formula in step S123 is: ; Among them, Output is the multi-head attention vector obtained after integration, is the feature vector of the h-th head, is the first pre-trained weight matrix; the Concat function is a feature vector concatenation function used to concatenate feature vectors. It is understandable that each "head" learns a different attention pattern, such as focusing on the onset of speech, energy changes, and sudden noise. By integrating feature vectors from multiple heads, this application can capture information more comprehensively and accurately and recognize speech.

[0070] According to some embodiments of the present application, step S124 is further described, wherein voice activity recognition processing is performed based on the multi-head attention vector at the current moment to obtain a voice activity probability sequence, including but not limited to steps S1241 to S1242.

[0071] Step S1241: Obtain the multi-head attention vector at the current moment.

[0072] Step S1242: Decode the multi-head attention vector at the current moment to obtain a speech activity probability sequence; the speech activity probability sequence includes: the speech existence probability corresponding to the data frames at different moments; the speech existence probability is used to characterize the probability of speech existing in the data frame at the corresponding moment, and the value of the speech existence probability is 0 to 1.

[0073] Specifically, the formula for obtaining the voice activity probability sequence is: Where P is the speech activity probability sequence. The speech presence probability in the speech activity probability sequence represents the probability of speech in the data frame at the corresponding time. The speech presence probability ranges from 0 to 1. Decode() is the decoding function. Output is the integrated multi-head attention vector.

[0074] According to some embodiments of the present application, step S125 implements signal conversion processing based on threshold judgment; step S125 is further explained, wherein a threshold judgment is performed on the voice activity probability sequence, and based on the judgment result obtained, the voice activity probability sequence is converted into a voice activity signal, and the voice activity signal is output, including but not limited to steps S1251 to S1253.

[0075] Step S1251: Obtain the speech presence probabilities corresponding to data frames at different times from the speech activity probability sequence.

[0076] Step S1252: Compare the speech existence probability with the preset threshold information frame by frame to determine the value of the data frame at the moment corresponding to the speech existence probability.

[0077] Step S1253: After the frame-by-frame comparison is completed, the voice activity signal is output in time sequence according to the values of the data frames at each moment.

[0078] According to some embodiments of the present application, step S1252 is further explained. Step S1252: comparing the probability of speech existence with the preset threshold information frame by frame to determine the value of the data frame at the moment corresponding to the probability of speech existence, including: when the preset threshold information is a single-value threshold, comparing the probability of speech existence with the single-value threshold frame by frame to obtain a first comparison result, and determining the value of the data frame at the moment corresponding to the probability of speech existence according to the first comparison result; or, when the preset threshold information includes: a first threshold value and a second threshold value greater than the first threshold value; comparing the probability of speech existence with the first threshold value and the second threshold value frame by frame to obtain a second comparison result, and determining the value of the data frame at the moment corresponding to the probability of speech existence according to the second comparison result.

[0079] It is understandable that the embodiment of the present application performs threshold judgment on the voice activity probability sequence in step S1252, and converts the voice activity probability sequence into a voice activity signal (ie, VAD signal) as follows: Specifically, the VAD signal can only take values of 0 and 1. 0 represents a low level, indicating that there is no speech in the current frame; 1 represents a high level, indicating that there is speech in the current frame. Single-valued or binary thresholds are commonly used to determine the speech activity probability sequence.

[0080] For example, when a single-value threshold is used and the single-value threshold = 0.5, the value of the data frame at the corresponding moment when the probability of speech existence is greater than or equal to 0.5 is determined to be 1; the value of the data frame at the corresponding moment when the probability of speech existence is less than 0.5 is determined to be 0; after the frame-by-frame comparison is completed, the voice activity signal is output frame by frame in chronological order according to the value of the data frame at each moment.

[0081] For example, when a binary threshold is used, that is, the first threshold value = 0.4, the second threshold value = 0.6; the value of the data frame at the corresponding moment when the probability of speech existence is greater than or equal to 0.6 is determined to be 1; the value of the data frame at the corresponding moment when the probability of speech existence is less than or equal to 0.4 is determined to be 0; for the data frame at the corresponding moment when the probability of speech existence is less than 0.6 and greater than 0.4, the value of the data frame of the previous frame can be referred to for determination; after the frame-by-frame comparison is completed, the voice activity signal is output frame by frame in chronological order according to the value of the data frame at each moment.

[0082] It is understandable that the preset threshold information can be a single-value threshold; or the preset threshold information includes a first threshold value and a second threshold value; as long as the preset threshold information can be used to implement threshold judgment, this application does not limit the specific form of the preset threshold information.

[0083] Through step S1251 to step S1252, the preset threshold information is used to perform threshold judgment on the voice activity probability sequence to generate a voice activity signal, so as to facilitate subsequent power consumption feedback control based on the obtained voice activity signal.

[0084] Through step S121 to step S125, by using the neural network voice activity detection model to directly perform voice activity detection on the oversampled initial audio sequence, the detection delay is reduced, which is conducive to improving the real-time performance of subsequent voice interaction.

[0085] Specifically, in step S130 , the feedback control signal includes a first control signal of a high level and a second control signal of a low level.

[0086] According to some embodiments of the present application, the level state of the voice activity signal includes: a high level; further explaining step S130, step S130: the power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module, and generates a feedback control signal to control the working mode of the signal acquisition module, including but not limited to steps S131 to S132.

[0087] Step S131: When the level state of the voice activity signal is high, the power consumption feedback control module generates a high-level first control signal in response to the high level.

[0088] Step S132: Control the signal acquisition module to enter the normal mode according to the first control signal.

[0089] According to some embodiments of the present application, the level state of the voice activity signal also includes: low level; further explaining step S130, step S130: the power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module, and generates a feedback control signal to control the working mode of the signal acquisition module, including but not limited to steps S133 to S135.

[0090] Step S133: When the level state of the voice activity signal is low level, the power consumption feedback control module continuously monitors the voice activity signal to obtain a low level duration.

[0091] Step S134: When the low-level duration exceeds a preset time threshold, a second low-level control signal is generated to control the signal acquisition module to enter a low-power consumption mode.

[0092] Step S135: When the low-level duration does not exceed the preset time threshold, the high-level first control signal is continuously output, and the control signal acquisition module remains in the normal mode.

[0093] Through steps S131 to S135, valid voice segments are identified based on the voice activity signal, and the signal acquisition module is controlled to operate in normal mode to acquire an oversampled initial audio sequence to ensure the normal operation of the detection system. Power consumption feedback control is performed through the obtained voice activity signal, and the operating mode of the signal acquisition module is dynamically adjusted, thereby effectively reducing the overall power consumption of the system.

[0094] This application does not use ADC threshold design, but introduces a power consumption feedback control mechanism: the ADC itself has two working modes, "low power" and "normal". When the VAD detects the presence of voice, the ADC switches from the "low power" mode to the "normal" mode; when the VAD does not detect voice, the ADC is in the "low power" mode. Since the VAD algorithm implemented by the large model itself has a very low false detection rate, mode switching will not be performed frequently, and the system power consumption can be effectively improved. If the ADC and the VAD algorithm implemented by the large model are integrated into one chip, the ADC will continue to generate audio data, and send it together with the VAD signal to the subsequent general computing platform for processing, which allows the subsequent algorithm to have a strong degree of freedom.

[0095] It's important to emphasize that this application separates the VAD function from the complex voice processing system and combines it with the ADC, integrating it into a single chip to form a universal voice front-end chip. This decouples front-end conversion and back-end processing while ensuring high accuracy and low power consumption, significantly improving versatility and ease of use. Users can choose ADCs with different signal-to-noise ratios or CPUs with different computing capabilities, without binding the two.

[0096] Take an example, combined with Figure 8 The following describes the specific process of power consumption feedback control. After receiving a voice activity signal, the system first determines whether the signal level is 1 (high). If so, the signal acquisition module enters normal operation mode. If not, the system monitors the duration of the low level. A determination is made as to whether the low level duration has expired. If so, the signal acquisition module enters low power mode. If not, the signal acquisition module enters normal operation mode.

[0097] like Figure 9As shown, when the initial voice signal includes valid voice segments and invalid noise segments, after the voice activity detection of the present application, a voice activity signal is obtained. The voice activity signal is high for valid voice segments and low for invalid noise segments. Based on this, the power consumption feedback control module generates a high-level first control signal in response to the high level of the voice activity signal, and controls the signal acquisition module to enter normal mode based on the first control signal. In response to the low level of the voice activity signal, the power consumption feedback control module continuously monitors the voice activity signal to obtain the low-level duration. When the low-level duration exceeds a preset time threshold, the power consumption feedback control module generates a low-level second control signal to control the signal acquisition module to enter low-power mode. When the low-level duration does not exceed the preset time threshold, the power consumption feedback control module continuously outputs the high-level first control signal to control the signal acquisition module to remain in normal mode. In this way, the operating mode of the signal acquisition module is flexibly adjusted based on the voice activity signal, ensuring normal processing of valid voice segments while also causing the signal acquisition module to enter low-power mode when receiving invalid noise segments, effectively reducing the overall power consumption of the system.

[0098] The analog-to-digital converter (ADC) itself has two operating modes: low-power mode and normal mode. When voice is detected (i.e., the VAD signal changes from low level to high level), the ADC is switched from low-power mode to normal mode. When the ADC is in normal mode, it continuously monitors the VAD level (high or low level, high level is "triggered" state, low level is "untriggered" state). Once the VAD is in the untriggered state for more than the preset time, the ADC returns from the "normal" working mode to the "low-power" working mode.

[0099] It is understandable that the input of the power consumption feedback control module is the VAD signal (word level), and the output is an equivalent feedback control signal ("sentence" level), which is used to control the working mode of the analog-to-digital converter ADC in the signal acquisition module to achieve the purpose of low power consumption of the system. Obviously, due to the diversity of speech, especially the different speaking speeds of speakers, the basic unit of VAD marking is the "word" level. If the "word" level VAD is directly fed back to control the working mode of the ADC, frequent switching will occur. Although the embodiment of the present application allows online switching of the working mode, frequent switching will cause the signal-to-noise ratio of the voice signal to fluctuate greatly. From the perspective of ensuring signal stability, it is hoped that the signal-to-noise ratio will be maintained as unchanged as possible. Frequent switching cannot further significantly improve power consumption. Therefore, considering all factors, it is more balanced in all aspects to feed back the feedback control signal at the "sentence" level to control the ADC working mode.

[0100] In one embodiment, if Figure 10As shown, before performing the voice activity signal detection method shown in steps S110 to S130, the voice activity signal detection method further includes: first, obtaining oversampled data from a historical database and designing an initial voice activity detection model to be trained; second, performing model training processing on the designed initial voice activity detection model to be trained based on the oversampled data; then, performing parameter optimization processing on the trained voice activity detection model to be evaluated; and then, performing parameter evaluation processing on the optimized voice activity detection model that has undergone parameter optimization processing, the parameter evaluation processing being: judging whether the parameter evaluation passes; if so, outputting the target neural network training model, and hardwareizing the neural network training model to complete pre-training; if not, re-performing the model training processing, parameter optimization processing, and parameter evaluation processing until the target neural network training model is obtained and hardwareized to complete pre-training.

[0101] This application implements VAD by moving the implementation of the VAD function forward to the audio data conversion stage. This is done by using an optimized large speech model, after the ADC converts the sound signal into high-speed audio data but before it is extracted into low-speed audio data. The large speech model used has been pre-trained in various complex scenarios and can achieve very high accuracy for human voices.

[0102] Specifically, model design is generally based on the Transformer architecture, with the number of parameters determined by the target model performance. Once the data and model are in place, model training begins. The trained model undergoes further parameter optimization, typically using a suitable loss function to measure model performance and error. An optimization algorithm is then used to minimize the loss function, thereby updating the model parameters. After training is complete, the model is hardware-implemented using an FPGA or ASIC. Typically, certain parameters are retained for fine-tuning to adapt to specific scenarios.

[0103] Those skilled in the art will appreciate that all or some of the steps and systems in the methods disclosed above may be implemented as software, firmware, hardware, or a suitable combination thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit.

[0104] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the present application.

Claims

1. A method for detecting a voice activity signal, characterized in that: The invention is applied to a voice activity signal detection system, the system comprising: a neural network voice activity detection module, a signal acquisition module and a power consumption feedback control module in communication with the neural network voice activity detection module; the power consumption feedback control module is also in communication with the signal acquisition module; wherein the neural network voice activity detection module comprises: a pre-trained neural network voice activity detection model; The method comprises: The neural network voice activity detection module receives at least one oversampled initial audio sequence collected by the signal acquisition module; The neural network voice activity detection model sequentially performs data preprocessing, feature vector encoding, vector fusion, voice activity recognition, and signal conversion based on the oversampled initial audio sequence to obtain a voice activity signal. The voice activity recognition process is used to decode the output multi-head attention vector at the current moment to obtain a voice activity probability sequence. The signal conversion process is used to perform threshold judgment on the voice activity probability sequence, convert the voice activity probability sequence into a voice activity signal based on the obtained judgment result, and output the voice activity signal. The power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module and generates a feedback control signal to control the working mode of the signal acquisition module; the working mode includes: low power consumption mode and normal mode.

2. The voice activity signal detection method according to claim 1, wherein: The level state of the voice activity signal includes: high level; The power consumption feedback control module monitors the level of the voice activity signal transmitted by the neural network voice activity detection module and generates a feedback control signal to control the working mode of the signal acquisition module, including: When the level state of the voice activity signal is: high level, the power consumption feedback control module generates a high-level first control signal in response to the high level; The signal acquisition module is controlled to enter a normal mode according to the first control signal.

3. The voice activity signal detection method according to claim 1, wherein: The level state of the voice activity signal also includes: low level; The power consumption feedback control module monitors the level of the voice activity signal transmitted by the neural network voice activity detection module and generates a feedback control signal to control the working mode of the signal acquisition module, including: When the level state of the voice activity signal is: low level, the power consumption feedback control module continuously monitors the voice activity signal to obtain a low level duration; When the low level duration exceeds a preset time threshold, a low-level second control signal is generated to control the signal acquisition module to enter the low power consumption mode; When the low level duration does not exceed the preset time threshold, the high level first control signal is continuously output to control the signal acquisition module to remain in the normal mode.

4. The voice activity signal detection method according to claim 1, wherein: The neural network voice activity detection model includes: a voice input layer, a feature vector encoding layer, a feature integration layer, a feature vector decoding and recognition layer, and an output layer connected in sequence; The neural network voice activity detection model sequentially performs data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing, and signal conversion processing based on the oversampled initial audio sequence to obtain a voice activity signal, including: The oversampled initial audio sequence is subjected to data preprocessing through the speech input layer to obtain an input matrix; wherein the input matrix includes a plurality of elements, and different elements represent feature vectors corresponding to different frame times; Performing feature vector encoding processing according to the input matrix through the feature vector encoding layer to obtain at least one feature vector; Through the feature integration layer, a vector fusion process is performed on at least one feature vector according to a pre-obtained first weight matrix and a preset integration formula to obtain a multi-head attention vector at the current moment; Performing voice activity recognition processing based on the multi-head attention vector at the current moment through the feature vector decoding and recognition layer to obtain a voice activity probability sequence; The output layer performs threshold judgment on the voice activity probability sequence, converts the voice activity probability sequence into a voice activity signal based on the obtained judgment result, and outputs the voice activity signal.

5. The voice activity signal detection method according to claim 4, characterized in that: The voice activity recognition process is performed based on the multi-head attention vector at the current moment to obtain a voice activity probability sequence, including: Obtain the multi-head attention vector at the current moment; Decode the multi-head attention vector at the current moment to obtain the voice activity probability sequence; the voice activity probability sequence includes: the voice existence probability corresponding to the data frames at different moments; the voice existence probability is used to characterize the probability of the existence of voice in the data frame at the corresponding moment, and the value of the voice existence probability is 0 to 1.

6. The voice activity signal detection method according to claim 5, characterized in that: The performing threshold judgment on the voice activity probability sequence, converting the voice activity probability sequence into a voice activity signal based on the obtained judgment result, and outputting the voice activity signal includes: Obtaining speech presence probabilities corresponding to data frames at different times from the speech activity probability sequence; Comparing the speech presence probability with preset threshold information frame by frame to determine the value of the data frame at the moment corresponding to the speech presence probability; After the frame-by-frame comparison is completed, the voice activity signal is output in time sequence according to the values of the data frames at each moment.

7. The voice activity signal detection method according to claim 6, characterized in that: The comparing the speech presence probability with preset threshold information frame by frame to determine the value of the data frame at the moment corresponding to the speech presence probability includes: When the preset threshold information is a single-value threshold, comparing the speech presence probability with the single-value threshold frame by frame to obtain a first comparison result, and determining a value of the data frame at the time corresponding to the speech presence probability according to the first comparison result; Alternatively, when the preset threshold information includes: a first threshold value and a second threshold value greater than the first threshold value; the probability of speech existence is compared with the first threshold value and the second threshold value frame by frame to obtain a second comparison result, and the value of the data frame at the moment corresponding to the probability of speech existence is determined according to the second comparison result.

8. A voice activity signal detection system, characterized in that: include: a neural network voice activity detection module, a signal acquisition module and a power consumption feedback control module communicatively connected to the neural network voice activity detection module; The power consumption feedback control module is also communicatively connected to the signal acquisition module; wherein the neural network voice activity detection module includes: a pre-trained neural network voice activity detection model; the signal acquisition module, the neural network voice activity detection module, and the power consumption feedback control module cooperate with each other to enable the voice activity signal detection system to implement the voice activity signal detection method according to any one of claims 1 to 7.

9. The voice activity signal detection system according to claim 8, characterized in that: The neural network voice activity detection model includes: a voice input layer, a feature vector encoding layer, a feature integration layer, a feature vector decoding and recognition layer, and an output layer connected in sequence.

10. The voice activity signal detection system according to claim 8, wherein: The signal acquisition module includes: at least one audio acquisition unit, each of which is used to acquire an oversampled initial audio sequence; and enters a low power consumption mode or a normal mode under the control of the power consumption feedback control module.

Images