Voice activity signal detection method and voice activity signal detection system

Abstract

The application discloses a voice activity signal detection method and a voice activity signal detection system, and relates to the field of audio signal processing, wherein the method comprises the following steps: a neural network voice activity detection module receives at least one over-sampled initial audio sequence collected by a signal collection module; through a neural network voice activity detection model, data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing and signal conversion processing are sequentially performed based on the over-sampled initial audio sequence to obtain a voice activity signal; a power consumption feedback control module monitors the level state of the voice activity signal input by the neural network voice activity detection module, generates a feedback control signal to control the working mode of the signal collection module; the working mode comprises a low-power mode and a normal mode. The method can reduce the detection delay, is beneficial to improving the real-time performance of subsequent voice interaction, and reduces the overall power consumption of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio signal processing, in particular to a voice activity signal detection method and a voice activity signal detection system. BACKGROUND

[0002] Speech is the most basic and natural way of human-computer interaction, and efficient and accurate speech processing is the most important technical link of speech interaction. Voice activity detection (VAD) is an important part before speech processing, and after identifying the effective speech segment and the invalid noise segment through voice activity detection, the subsequent speech processing is performed on the effective speech segment. Common speech processing includes speech coding, speech noise reduction, speech recognition, speech enhancement, etc. The current mainstream voice activity detection technical solution is: sound signals are collected through a microphone array, and the collected sound signals are converted into multi-channel audio data by a multi-channel ADC (analog-to-digital converter). Each channel of audio data is composed of an effective speech segment and an invalid noise segment. The multi-channel audio data is transmitted to a general computing platform through a special data interface. The general computing platform first runs a VAD algorithm to process the multi-channel audio data and generates a VAD signal, and the subsequent speech algorithm performs more complex speech processing based on the VAD signal. In this way, the entire system needs to be in a working state at all times to respond to voice commands in a timely manner, which results in high overall system power consumption. In addition, the sampling frequency of the original speech data is generally low, usually 16 kHz, and when a slightly complex VAD algorithm is used on the general computing platform, more time periods are required for operation, which will increase the data processing delay and result in poor real-time performance of speech interaction. SUMMARY

[0003] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides a voice activity signal detection method and a voice activity signal detection system, which can detect the over-sampled initial audio sequence to obtain a voice activity signal, thereby reducing the detection delay and improving the real-time performance of subsequent speech interaction. The power consumption feedback control is performed through the voice activity signal, which is conducive to reducing the overall system power consumption.

[0004] In a first aspect, an embodiment of the present application provides a voice activity signal detection method applied to a voice activity signal detection system. The system includes a neural network voice activity detection module, a signal acquisition module and a power consumption feedback control module in communication with the neural network voice activity detection module. The power consumption feedback control module is also in communication with the signal acquisition module. The neural network voice activity detection module includes a pre-trained neural network voice activity detection model.

[0005] The method includes:

[0006] The neural network voice activity detection module receives at least one over-sampled initial audio sequence collected by the signal collection module;

[0007] Through the neural network voice activity detection model, data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing and signal conversion processing are sequentially performed based on the over-sampled initial audio sequence, to obtain a voice activity signal; wherein the voice activity recognition processing is used for decoding according to the output multi-head attention vector at the current time to obtain a voice activity probability sequence; the signal conversion processing is used for threshold judgment on the voice activity probability sequence, based on the obtained judgment result, converting the voice activity probability sequence into a voice activity signal, and outputting the voice activity signal;

[0008] The power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module, generates a feedback control signal to control the working mode of the signal collection module; the working mode includes: low power consumption mode and normal mode.

[0009] In a second aspect, the present application provides a voice activity signal detection system, comprising: a neural network voice activity detection module, a signal collection module and a power consumption feedback control module in communication with the neural network voice activity detection module; the power consumption feedback control module is also in communication with the signal collection module; wherein the neural network voice activity detection module comprises: a pre-trained neural network voice activity detection model; the signal collection module, the neural network voice activity detection module, the power consumption feedback control module cooperate with each other, so that the voice activity signal detection system realizes the voice activity signal detection method as described in any one of the first aspect embodiments.

[0010] The voice activity signal detection system includes a neural network voice activity detection module, a signal acquisition module in communication with the neural network voice activity detection module, and a power consumption feedback control module in communication with the signal acquisition module. The neural network voice activity detection module includes a pre-trained neural network voice activity detection model. During voice activity signal detection processing using the voice activity signal detection system, the neural network voice activity detection module first receives at least one over-sampled initial audio sequence collected by the signal acquisition module. The over-sampled initial audio sequence is directly collected to lay a data foundation for subsequent rapid voice activity detection. Then, the neural network voice activity detection model is used to sequentially perform data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing, and signal conversion processing based on the over-sampled initial audio sequence to obtain a voice activity signal. The voice activity recognition processing is used to decode the output multi-head attention vector at the current time to obtain a voice activity probability sequence. The signal conversion processing is used to perform threshold judgment on the voice activity probability sequence, convert the voice activity probability sequence into a voice activity signal based on the obtained judgment result, and output the voice activity signal. The neural network voice activity detection model is used to directly perform voice activity detection on the over-sampled initial audio sequence, which reduces the detection delay and improves the real-time performance of subsequent voice interaction. Finally, the power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module, generates a feedback control signal to control the working mode of the signal acquisition module, and the working mode includes a low-power mode and a normal mode. The power consumption feedback control is performed based on the obtained voice activity signal to dynamically adjust the working mode of the signal acquisition module, thereby effectively reducing the overall power consumption of the system. In other words, the voice activity signal is obtained by detecting the over-sampled initial audio sequence, and the power consumption feedback control is performed based on the voice activity signal, thereby reducing the detection delay and improving the real-time performance of subsequent voice interaction, and effectively reducing the overall power consumption of the system. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 FIG. 1 is a system architecture schematic diagram of a voice activity signal detection system for performing a voice activity signal detection method according to an embodiment of the present application;

[0012] Figure 2 FIG. 2 is a schematic diagram of a conventional voice activity signal detection architecture according to an embodiment of the present application;

[0013] Figure 3 FIG. 3 is a specific schematic diagram of a voice activity signal detection system according to an embodiment of the present application;

[0014] Figure 4 is a specific structural schematic diagram of an analog-to-digital converter provided by an embodiment of the present application;

[0015] Figure 5 is a specific structural schematic diagram of a neural network voice activity detection model provided by an embodiment of the present application;

[0016] Figure 6 is a specific structural schematic diagram of a voice activity signal detection system implemented based on an ASIC chip provided by an embodiment of the present application;

[0017] Figure 7 is a flowchart of a voice activity signal detection method provided by an embodiment of the present application;

[0018] Figure 8 is a flowchart of a power consumption feedback control process provided by an embodiment of the present application;

[0019] Figure 9 is a timing schematic diagram of a voice activity signal and a feedback control signal provided by an embodiment of the present application;

[0020] Figure 10 is a flowchart of a process of training a neural network voice activity detection model provided by an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of the present application clearer, further detailed descriptions will be given to the present application in combination with the accompanying drawings and embodiments.

[0022] It should be noted that, although a logical order is shown in the flowchart in the description of the present application, in some cases, the steps shown or described can be performed in an order different from that in the flowchart. In the description of the present application, the meaning of “one or more” is one or more, and the meaning of “two or more” is two or more. The description of “first”, “second” is only for distinguishing technical features for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the sequence of indicated technical features.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0024] First, some terms involved in the present application are explained:

[0025] Voice Activity Detection (VAD) is a widely used technology in speech processing. After identifying the valid speech segments and invalid noise segments through voice activity detection, subsequent speech processing is performed on the valid speech segments. Common speech processing includes: speech coding, speech recognition, speech noise reduction, speech enhancement, etc. In speech coding, speech activity frames need to be identified and coded to save about 50% transmission bandwidth; in speech recognition, speech segments containing human voice need to be extracted for speech recognition algorithm processing to save system operation resources; in speech noise reduction / enhancement, non-speech segments need to be removed or speech segments need to be enhanced. Although VAD has different focuses in different application scenarios, the essence is to detect and distinguish speech segments and non-speech segments. Accuracy and delay are two important evaluation dimensions, and even directly affect the experience of human-computer interaction. Among them, the accuracy is evaluated from two dimensions: false alarm rate and missed detection rate. The false alarm rate refers to the ratio of non-speech segments detected as speech segments, and the missed detection rate refers to the ratio of speech segments missed as non-speech segments. High accuracy means low false alarm rate and low missed detection rate.

[0026] The application provides a voice activity signal detection method and a voice activity signal detection system, and relates to the field of audio signal processing. The method comprises the following steps: a neural network voice activity detection module receives at least one over-sampled initial audio sequence collected by a signal collection module; through a neural network voice activity detection model, data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing and signal conversion processing are sequentially performed based on the over-sampled initial audio sequence to obtain a voice activity signal; a power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module, generates a feedback control signal to control the working mode of the signal collection module; the working mode comprises a low-power mode and a normal mode. The detection delay can be reduced, which is conducive to improving the real-time performance of subsequent voice interaction; and the overall power consumption of the system is reduced.

[0027] The embodiments of the application are further described below with reference to the drawings.

[0028] As Figure 1As shown, the voice activity signal detection system 100 comprises: a neural network voice activity detection module 120, a signal acquisition module 110 and a power consumption feedback control module 130 connected in communication with the neural network voice activity detection module 120; the power consumption feedback control module 130 is also connected in communication with the signal acquisition module 110; wherein the neural network voice activity detection module 120 comprises: a pre-trained neural network voice activity detection model; the signal acquisition module 110, the neural network voice activity detection module 120 and the power consumption feedback control module 130 cooperate with each other, so that the voice activity signal detection system realizes the voice activity signal detection method as in the embodiments of the present application. The voice activity signal detection system provided in the embodiments of the present application can detect the over-sampled initial audio sequence to obtain a voice activity signal, thereby reducing the detection delay and being conducive to improving the real-time performance of subsequent voice interaction; and the power consumption feedback control is performed through the voice activity signal, which is conducive to reducing the overall power consumption of the system.

[0029] Specifically, in the voice activity signal detection system 100, the neural network voice activity detection module 120 receives at least one over-sampled initial audio sequence collected by the signal acquisition module 110; through the neural network voice activity detection model, data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing and signal conversion processing are sequentially performed based on the over-sampled initial audio sequence, so as to obtain a voice activity signal; the power consumption feedback control module 130 monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module 120, generates a feedback control signal to control the working mode of the signal acquisition module 110; the working mode includes: a low-power mode and a normal mode. Through the power consumption feedback control of the obtained voice activity signal, the working mode of the signal acquisition module is dynamically adjusted, so as to effectively reduce the overall power consumption of the system.

[0030] It can be understood that the traditional voice activity detection architecture such as Figure 2As shown, the traditional voice activity detection architecture includes one or more signal conversion paths, a downsampling module, a general computing platform and a memory; wherein the signal conversion path includes a microphone and an analog-to-digital converter ADC; specifically, the microphone, the analog-to-digital converter, the general computing platform and the memory are sequentially communicatively connected. It can be seen that the most commonly used voice activity detection method in the industry at present is: one or more signal conversion paths (including a microphone and an analog-to-digital converter) are used for collection, to generate oversampled audio data, the oversampled audio data is first extracted by the downsampling module to obtain standard audio data, and the rate of the standard audio data is generally 16 kHz; and then the standard audio data is sent to the general computing platform, and a complex VAD algorithm is run on the general computing platform to generate a VAD signal. The general computing platform and related storage can be any one or more forms independent of the voice front end, including CPU, NPU, GPU, FPGA and ASIC, etc., or can be integrated with the voice front end. In the process of using the traditional voice activity detection method, the entire detection system needs to be in a working state at all times to respond to voice commands in a timely manner, which leads to high power consumption of the overall system. Moreover, the sampling frequency of the original voice data processed by the detection system is generally low, generally 16 kHz, and when the general computing platform uses a slightly complex VAD algorithm, more time periods are needed for calculation, which will increase the data processing delay and lead to poor real-time performance of voice interaction. It can be seen that the shortcomings of using the traditional voice activity detection architecture for voice activity detection are high power consumption and large delay. The voice activity signal detection system provided in the embodiments of the present application first does not perform downsampling processing on the oversampled initial audio sequence to be processed using a standard sampling rate after collecting the oversampled initial audio sequence, but directly processes the collected oversampled initial audio sequence using a neural network voice activity detection module to output a voice activity signal, thereby omitting the downsampling process in the ADC conversion process, the signal collection and processing speed is faster, the overall voice activity detection processing delay is reduced, and the real-time performance of subsequent voice interaction is improved.

[0031] Further, in combination with Figure 3 The signal collection module 110 is further described. According to some embodiments of the present application, the signal collection module 110 includes at least one audio collection unit 111; each audio collection unit 111 is used to collect an oversampled initial audio sequence; and under the control of the power consumption feedback control module 130, enters a low-power mode or a normal mode.

[0032] In some embodiments, each audio acquisition unit 111 comprises a microphone 111A and an analog-to-digital converter ADC; the voice activity signal detection system 100 further comprises a signal input end 101; the microphone 111A is in communication connection with the signal input end 101, and the microphone 111A is configured to receive an analog signal input through the signal input end 101 and input the analog signal to the analog-to-digital converter ADC; the analog-to-digital converter ADC performs analog-to-digital conversion processing on the incoming analog signal to obtain an oversampled initial audio sequence. And input the oversampled initial audio sequence to the neural network voice activity detection module 120.

[0033] Specifically, in combination with Figure 4 , the analog-to-digital converter ADC is further described. In some embodiments, as shown in Figure 4 , the analog-to-digital converter ADC comprises a programmable gain amplifier 1121 and a Sigma-Delta modulator 1122 connected in sequence; wherein the input end of the programmable gain amplifier 1121 is connected with the output end of the microphone 111A, and the output end of the Sigma-Delta modulator 1122 is connected with the input end of the neural network voice activity detection module 120. Specifically, the programmable gain amplifier 1121 is configured to receive the analog signal (voltage signal) output by the microphone 111A, and output the amplified analog signal to the Sigma-Delta modulator 1122; the Sigma-Delta modulator 1122 modulates the amplified analog signal into an oversampled initial audio sequence (i.e. digital signal); and inputs the oversampled initial audio sequence to the neural network voice activity detection module 120.

[0034] Specifically, the programmable gain amplifier 1121 can amplify the analog signal (voltage signal) according to a given gain, and if the gain required by the system is 1, the programmable gain amplifier can be omitted; the Sigma-Delta modulator 1122 converts the amplified voltage signal into oversampled audio data at a fixed rate, and the conversion rate = standard audio rate * oversampling rate (OSR). For most speech recognition scenarios, the standard audio rate is 16 kHz, and the oversampling rate is related to the design, and the common values are 64, 128 and 256.

[0035] In the embodiments of the present application, through the cooperation of the microphone 111A and the analog-to-digital converter ADC in the signal acquisition module 110, the oversampled initial audio sequence is directly acquired, which lays a data foundation for subsequent rapid voice activity detection.

[0036] In another embodiment, the audio acquisition unit 111 can only comprise a digital microphone for outputting a digital signal. Specifically, the microphone for outputting a digital signal can be implemented by a silicon microphone.

[0037] It can be understood that the microphone and the analog-to-digital converter cooperate to output the collected sound signal as a digital signal, and the digital microphone can output the collected sound signal as a digital signal; it can be seen that when the audio acquisition unit adopts the microphone and the analog-to-digital converter or only adopts the digital microphone, the same effect can be achieved; therefore, the specific structure composition and implementation mode of the audio acquisition unit are not limited.

[0038] Further, the neural network voice activity detection module 120 is further described. The neural network voice activity detection module 120 includes: a neural network voice activity detection model obtained through pre-training; specifically, the network voice activity detection model is based on a multi-head self-attention mechanism.

[0039] As shown in Figure 5 According to some embodiments of the present application, the neural network voice activity detection module 120 includes: a neural network voice activity detection model 121, which includes: a voice input layer, a feature vector encoding layer, a feature integration layer, a feature vector decoding recognition layer and an output layer connected in sequence.

[0040] The voice input layer is used for data preprocessing of the oversampled initial audio sequence, and an input matrix is obtained accordingly; the input matrix includes a plurality of elements, and different elements represent feature vectors corresponding to different frame moments; the feature vector encoding layer is used for feature vector encoding processing according to the input matrix, and at least one feature vector is obtained accordingly; the feature integration layer is used for vector fusion processing of the at least one feature vector according to a first weight matrix obtained in advance and a preset integration formula, and a multi-head attention vector of the current moment is integrated; the feature vector decoding recognition layer is used for voice activity recognition processing according to the multi-head attention vector of the current moment, and a voice activity probability sequence is obtained; the output layer is used for threshold judgment of the voice activity probability sequence, and based on the obtained judgment result, the voice activity probability sequence is converted into a voice activity signal, and the voice activity signal is output.

[0041] Specifically, as shown in Figure 5As shown, the voice input layer includes a plurality of voice input units responsible for receiving a plurality of voice signals, each voice signal being represented as a separate sequence. Since the self-attention mechanism is not sensitive to position information, position encoding is introduced in the voice input layer to enable the neural network voice activity detection model to understand the timing information in the initial audio sequence input in subsequent processing. In addition, the voice input layer is responsible for data preprocessing of the input oversampled initial audio sequence to obtain an input matrix, which includes but is not limited to frame processing, raw feature extraction processing, dimension expansion processing, and position encoding processing. The input matrix includes a plurality of elements, with different elements representing feature vectors corresponding to different frame times; that is, the input matrix includes the basic features and timing information of the elements.

[0042] Specifically, as shown in Figure 5 The feature vector encoding layer includes a plurality of transformer encoders, one transformer encoder corresponding to one voice input unit and receiving one input matrix. In the feature vector encoding layer, based on the multi-head self-attention mechanism, each initial audio sequence is encoded into a high-dimensional feature vector. This provides a data basis for subsequent integrated multi-head attention vectors.

[0043] Specifically, the feature integration layer integrates the multi-head self-attention mechanism to weight and integrate the high-dimensional feature vectors output by each encoder using the first weight matrix obtained in advance to obtain a target integrated vector (multi-head attention vector) at the current time, realizing the extraction and understanding of global context information.

[0044] Specifically, as shown in Figure 5 The feature vector decoding layer includes a lightweight time classification classifier that decodes and generates a voice activity probability sequence in combination with the multi-head attention vectors of the previous and current times.

[0045] Specifically, in combination with Figure 3 and Figure 5 The output layer converts the voice activity probability sequence into a voice activity signal corresponding to the audio data. On the one hand, the voice activity signal is output to the power consumption feedback control module; on the other hand, the voice activity signal is output to the outside through the first output end 102 of the voice activity signal detection system. In another embodiment, the output layer can also be configured to output the oversampled initial audio data, or when the neural network voice activity detection model 121 also integrates a noise reduction function, the output layer can also be configured to output the oversampled audio data after noise reduction; which will not be repeated here.

[0046] It can be understood that through the mutual cooperation of the voice input layer, the feature vector encoding layer, the feature integration layer, the feature vector decoding recognition layer and the output layer, the collected oversampled initial audio sequence can be directly subjected to voice activity detection processing, and the voice activity signal can be quickly output, thereby reducing the detection delay and being conducive to improving the real-time performance of subsequent voice interaction.

[0047] According to some embodiments of the present application, the neural network voice activity detection module is a hardware module implemented in an FPGA or ASIC manner.

[0048] It can be understood that the neural network voice activity detection model is a pre-trained Transformer network, and the pre-training can fix most parameters, which can be implemented by a pure logic circuit. Unlike a software-based voice activity detection method, the neural network voice activity detection module of the embodiments of the present application is hardware after hardware, and does not require a general-purpose computing module such as a CPU or a DSP and a memory. In this way, the hardware cost is reduced.

[0049] According to some embodiments of the present application, the voice activity signal detection system further comprises a parameter adjustment module, which is electrically connected with the neural network voice activity detection module; the parameter adjustment module is used to adjust the parameters in the hardware neural network voice activity detection module.

[0050] It can be understood that in the process of pre-training the neural network model, the neural network model is generally trained by a large amount of multi-channel voice data in various environments. Therefore, a large number of parameters of the obtained neural network model after training are fixed, so that the neural network voice activity detection module including the neural network voice activity detection model can be hardware by FPGA or ASIC. It should be noted that in order to make the hardware neural network voice activity detection module have better adaptability and be better suitable for different specific scenes, some parameters in the neural network voice activity detection model can be selectively retained for fine adjustment.

[0051] In some embodiments, the down-sampling filter module is used to convert the low-precision oversampled data into high-precision standard sampling rate data, which is generally implemented by combining FIR filters and IIR filters. The current traditional voice recognition method generally converts the standard sampling rate in the cloud, and then performs voice recognition based on the converted standard sampling rate, which ensures the accuracy and timeliness of voice recognition. The embodiments of the present application perform down-sampling processing on the oversampled data output by the neural network voice activity detection model through the setting of the down-sampling filter module, so as to adapt to the subsequent voice recognition and meet the accuracy and timeliness of the subsequent voice recognition. Moreover, the multi-channel voice data does not need to be uploaded to the cloud, which greatly reduces the requirements for network bandwidth and hardware.

[0052] In some embodiments, as shown in FIG. 1, the voice activity signal detection system 100 further comprises a downsampling filter module 140, a first output end 102 and a second output end 103; an input end of the downsampling filter module 140 is connected with another output end of the neural network voice activity detection module 120, an output end of the downsampling filter module 140 is connected with the second output end 103, and the second output end 103 is configured to output the initial audio data after downsampling at the same time when the voice activity signal is outputted at the first output end 102. Alternatively, when the neural network voice activity detection module 120 is further integrated with a noise reduction function, the second output end 103 is configured to output the audio data after noise reduction and downsampling at the same time when the voice activity signal is outputted at the first output end 102. Figure 3

[0053] Further, the downsampling filter module 140 is configured to extract the oversampled audio data with low precision into standard sampling rate audio data with high precision, which is generally implemented by using a FIR filter or an IIR filter. At present, most voice processing is based on standard rate audio data, such as 16 KHz. The conventional voice recognition mode at present generally converts the standard sampling rate in the cloud, and then performs voice recognition based on the conversion into the standard sampling rate, so as to ensure the accuracy and timeliness of voice recognition. However, the embodiments of the present application set the downsampling filter module to perform downsampling processing on the oversampled data outputted by the neural network voice activity detection model, so as to adapt to the subsequent voice recognition, and meet the accuracy and timeliness of the subsequent voice recognition. Moreover, the multiple voice data does not need to be uploaded to the cloud, which greatly reduces the requirements for network bandwidth and hardware.

[0054] For example, as shown in FIG. 1, the voice activity signal detection system 100 further comprises a downsampling filter module 140, a first output end 102 and a second output end 103; an input end of the downsampling filter module 140 is connected with another output end of the neural network voice activity detection module 120, an output end of the downsampling filter module 140 is connected with the second output end 103, and the second output end 103 is configured to output the initial audio data after downsampling at the same time when the voice activity signal is outputted at the first output end 102. Alternatively, when the neural network voice activity detection module 120 is further integrated with a noise reduction function, the second output end 103 is configured to output the audio data after noise reduction and downsampling at the same time when the voice activity signal is outputted at the first output end 102. Figure 6 ​As shown, the voice activity signal detection system is implemented as an ASIC chip. The ASIC chip integrates two analog-to-digital converters ADCs for converting analog electrical signals from analog microphones MIC0 and MIC1 into digital signals; at the same time, it can also directly receive PDM (Pulse Density Modulation) signals from digital microphones DMIC#0 and DMIC#1. A first multiplexer mux0 connected to ADC0 and digital microphone DMIC#0 selects one of the converted digital signals and PDM signals and outputs it to a pre-trained neural network voice activity detection module; a second multiplexer mux1 connected to ADC1 and digital microphone DMIC#1 selects one of the converted digital signals and PDM signals and outputs it to a pre-trained neural network voice activity detection module. After processing by the neural network voice activity detection module, a voice activity signal and a route of oversampled data are generated, and this oversampled data is converted into standard sampling rate data by a downsampling filter module, and then output to other models through a standard audio IIS interface for subsequent speech recognition, semantic understanding, etc. It can be understood that the ASIC chip also reserves some adjustable parameters for the neural network voice activity detection module to adapt to special requirements in applications, and the adjustment of the parameters is realized through a parameter adjustment module and an IIC interface. Further, the ASIC chip also integrates necessary power supply, reference voltage VREF, clock signal, and power consumption feedback control modules.

[0055] In summary, the voice activity signal detection system provided by the embodiments of the present application separates the VAD function from the general computing platform, and advances the timing of implementing the VAD function to after the ADC output in the system architecture, runs a pre-trained neural network voice activity detection model based on the multi-head self-attention mechanism, directly processes the oversampled audio data and outputs the VAD signal (i.e., the voice activity signal), thereby realizing the VAD function; and has the advantages of high detection accuracy, low power consumption, small delay, and strong universality.

[0056] Those skilled in the art can understand that the system structure shown in the figure does not constitute a limitation on the embodiments of the present application, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0057] Those skilled in the art can understand that the system architecture and application scenarios described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, as the system architecture evolves and new application scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0058] Based on the above system structure, the following embodiments of the voice activity signal detection method of the present application are proposed.

[0059] As shown in the voice activity signal detection method, the voice activity signal detection method is applied to the voice activity signal detection system as shown in the voice activity signal detection system, and the voice activity signal detection system comprises: a neural network voice activity detection module, a signal acquisition module and a power consumption feedback control module in communication connection with the neural network voice activity detection module; the power consumption feedback control module is also in communication connection with the signal acquisition module; wherein the neural network voice activity detection module comprises: a pre-trained neural network voice activity detection model; the voice activity signal detection method can include but is not limited to steps S110 to S130. Figure 7 Figure 1 Step S110: The neural network voice activity detection module receives at least one over-sampled initial audio sequence collected by the signal acquisition module.

[0060] Step S120: Through the neural network voice activity detection model, data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing and signal conversion processing are sequentially performed based on the over-sampled initial audio sequence, to obtain a voice activity signal; wherein the voice activity recognition processing is used for decoding according to the output multi-head attention vector at the current time to obtain a voice activity probability sequence; the signal conversion processing is used for threshold judgment on the voice activity probability sequence, and based on the obtained judgment result, the voice activity probability sequence is converted into a voice activity signal, and the voice activity signal is output.

[0061] Step S130: The power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module, generates a feedback control signal to control the working mode of the signal acquisition module; the working mode includes: low power consumption mode and normal mode.

[0062] It can be understood that each over-sampled initial audio sequence is a separate input sequence, wherein the initial audio sequence comprises a plurality of elements, and each element is a multi-dimensional vector.

[0063] It can be understood that the level state of the voice activity signal includes: high level, low level.

[0064] It can be understood that the feedback control signal includes a first control signal (high level, value 1) and a second control signal (low level, value 0).

[0065] It can be understood that the feedback control signal includes a first control signal (high level, value 1) and a second control signal (low level, value 0).

[0066] ​It can be understood that, through the step S110 and the step S120, the oversampled initial audio sequence is directly processed, and the downsampling processing in the ADC conversion process is omitted in the step S110, so that the signal acquisition speed is faster, and the overall voice activity detection processing delay is smaller.

[0067] The data source processed by the embodiment of the application is the high-speed audio signal (oversampled initial audio sequence) before decimation, which naturally has a faster processing speed, so the delay is also lower, and finally the oversampled initial audio sequence is also subjected to decimation before being sent to a general computing platform, which is equivalent to that the voice activity signal is obtained by "advance processing", which can greatly reduce or even eliminate the need for data Buffer, and the overall hardware overhead is smaller.

[0068] In the process of using the voice activity signal detection system to perform voice activity signal detection processing through the steps S110 to S130, first, the neural network voice activity detection module receives at least one oversampled initial audio sequence collected by the signal acquisition module; the oversampled initial audio sequence is directly collected, which lays a data foundation for subsequent relatively fast voice activity detection; then, through the neural network voice activity detection model, data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing and signal conversion processing are sequentially performed based on the oversampled initial audio sequence to obtain a voice activity signal; wherein the voice activity recognition processing is used for decoding according to the output multi-head attention vector at the current time to obtain a voice activity probability sequence; the signal conversion processing is used for threshold judgment on the voice activity probability sequence, and based on the obtained judgment result, the voice activity probability sequence is converted into the voice activity signal, and the voice activity signal is output; by using the neural network voice activity detection model to directly perform voice activity detection on the oversampled initial audio sequence, the detection delay is reduced, which is conducive to improving the real-time performance of subsequent voice interaction; finally, the power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module, generates a feedback control signal to control the working mode of the signal acquisition module; the working mode includes: a low-power mode and a normal mode. Through the power consumption feedback control of the obtained voice activity signal, the working mode of the signal acquisition module is dynamically adjusted, so that the overall system power consumption can be effectively reduced. That is to say, the scheme of the embodiment of the application can detect the oversampled initial audio sequence to obtain the voice activity signal, so as to reduce the detection delay, which is conducive to improving the real-time performance of subsequent voice interaction; and through the power consumption feedback control of the voice activity signal, the overall system power consumption is effectively reduced.

[0069] According to some embodiments of the present application, a neural network voice activity detection model includes: a voice input layer, a feature vector encoding layer, a feature integration layer, a feature vector decoding and recognition layer, and an output layer connected in sequence; further explaining step S120, step S120: through the neural network voice activity detection model, based on the oversampled initial audio sequence, data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing, and signal conversion processing are performed in sequence to obtain a voice activity signal, including but not limited to steps S121 to S125.

[0070] Step S121: Preprocess the oversampled initial audio sequence through the speech input layer to obtain a corresponding input matrix; wherein the input matrix includes multiple elements, and different elements represent feature vectors corresponding to different frame times.

[0071] Step S122: performing feature vector encoding processing according to the input matrix through the feature vector encoding layer to obtain at least one feature vector.

[0072] Step S123: Through the feature integration layer, vector fusion processing is performed on at least one feature vector according to the pre-obtained first weight matrix and the preset integration formula to integrate and obtain the multi-head attention vector at the current moment.

[0073] Step S124: Through the feature vector decoding recognition layer, voice activity recognition processing is performed according to the multi-head attention vector at the current moment to obtain a voice activity probability sequence.

[0074] Step S125: performing threshold judgment on the voice activity probability sequence through the output layer, converting the voice activity probability sequence into a voice activity signal based on the obtained judgment result, and outputting the voice activity signal.

[0075] Further describing step S121, data preprocessing is performed on the input oversampled initial audio sequence. This data preprocessing includes, but is not limited to, framing, original feature extraction, dimensionality expansion, and position encoding. The input matrix obtained after data preprocessing includes multiple elements, each representing a feature vector corresponding to a different frame time; that is, the input matrix includes the basic features and timing information of the elements.

[0076] Specifically, the input oversampled initial audio sequence is preprocessed to obtain the input matrix X, which can be expressed as:

[0077] ; where each element of the input matrix X It is a d-dimensional vector, i represents different frame moments, d represents the feature dimension, and T represents the T-th frame, that is, the number of frames.

[0078] Further illustrate step S122, wherein the feature vector encoding processing is performed according to the input matrix, and at least one feature vector is obtained, including but not limited to steps S1221 to S1224.

[0079] Step S1221: performing linear transformation on the input matrix according to the preset weight matrix obtained through pre-training to obtain a projection matrix; the projection matrix includes a query matrix Q, a key matrix K and a value matrix V. Specifically, ; wherein, is a second weight matrix obtained through training, is a third weight matrix obtained through training, is a fourth weight matrix obtained through training. Specifically, the second weight matrix, the third weight matrix and the fourth weight matrix are all from the same characteristic matrix X of the previous layer, that is, the input initial audio sequence is linearly mapped multiple times to obtain different projection matrices.

[0080] Step S1222: calculating the dot product of the query matrix Q and the key matrix K, and dividing the dot product by the dimension of the key matrix K to obtain an attention score matrix. The calculation expression of the attention score matrix is:

[0081] .

[0082] Step S1223: calculating the attention output matrix by using an activation parameter to obtain an attention weight matrix. That is, applying an activation function (i.e., a Softmax function) to each row of the attention score matrix to obtain the attention weight matrix. The calculation expression of the attention weight matrix is:

[0083] .

[0084] Step S1224: performing weighted summation processing on the value matrix V according to the attention weight matrix to obtain a feature vector; the feature vector is an output matrix. The expression of the output matrix is:

[0085] .

[0086] Through steps S1221 to S1224, through the multi-head self-attention mechanism, on the basis of the self-attention mechanism, multiple sets of linear transformations are added, so that the neural network voice activity detection model can pay attention to different positions in different subspaces in parallel, thereby capturing more information and achieving better detection effect.

[0087] Further illustrate step S123, specifically, step S123 is used for splicing the feature vectors output under the multi-head attention mechanism, and obtaining a multi-head attention vector after linear transformation. The preset integration formula in step S123 is:

[0088] ;

[0089] wherein, Output is the multi-head attention vector obtained after integration, is the feature vector of the hth head, is the first weight matrix obtained by pre-training; the Concat function is a feature vector splicing function used for splicing feature vectors. It can be understood that each "head" learns different attention patterns, such as paying attention to the start of the voice, energy change, burst noise, etc. After integrating the feature vectors of multiple heads, the present application can more comprehensively and accurately capture information and identify the voice.

[0090] According to some embodiments of the present application, step S124 is further described, wherein the voice activity recognition processing is performed according to the multi-head attention vector at the current moment to obtain a voice activity probability sequence, including but not limited to steps S1241 to S1242.

[0091] Step S1241: Obtain the multi-head attention vector at the current moment.

[0092] Step S1242: Decode the multi-head attention vector at the current moment to obtain a voice activity probability sequence; the voice activity probability sequence includes voice existence probabilities corresponding to data frames at different moments; the voice existence probability is used to represent the probability size of the existence of voice corresponding to the data frame at the moment, and the voice existence probability has a value of 0 to 1.

[0093] Specifically, the formula expression for obtaining the voice activity probability sequence is:

[0094] ; wherein, P is the voice activity probability sequence; the voice existence probability in the voice activity probability sequence is used to represent the probability size of the existence of voice corresponding to the data frame at the moment, and the voice existence probability has a value of 0 to 1. Decode() is a decoding function; Output is the multi-head attention vector obtained after integration.

[0095] According to some embodiments of the present application, step S125 implements signal conversion processing based on threshold judgment; step S125 is further described, wherein the voice activity probability sequence is subjected to threshold judgment, the voice activity probability sequence is converted into a voice activity signal based on the obtained judgment result, and the voice activity signal is output, including but not limited to steps S1251 to S1253.

[0096] Step S1251: Obtain the voice existence probability corresponding to the data frame at different moments from the voice activity probability sequence.

[0097] Step S1252: compare the voice presence probability with preset threshold information frame by frame, and determine the value of the data frame corresponding to the time of the voice presence probability.

[0098] Step S1253: after the frame-by-frame comparison, output the voice activity signal according to the value of the data frame at each time in time sequence.

[0099] According to some embodiments of the present application, step S1252 is further described as follows: step S1252: compare the voice presence probability with preset threshold information frame by frame, and determine the value of the data frame corresponding to the time of the voice presence probability, including: when the preset threshold information is a single threshold, compare the voice presence probability with the single threshold frame by frame to obtain a first comparison result, and determine the value of the data frame corresponding to the time of the voice presence probability according to the first comparison result; or when the preset threshold information includes a first threshold value and a second threshold value greater than the first threshold value, compare the voice presence probability with the first threshold value and the second threshold value frame by frame to obtain a second comparison result, and determine the value of the data frame corresponding to the time of the voice presence probability according to the second comparison result.

[0100] It can be understood that, according to step S1252, the voice activity probability sequence is threshold judged, and the voice activity probability sequence is converted into a voice activity signal (i.e. a VAD signal) as follows:

[0101] Specifically, the value of the VAD signal is only 0 and 1, 0 represents a low level and is used to indicate that there is no voice in the current frame, and 1 represents a high level and is used to indicate that the current frame is voice. A single threshold or a binary threshold is commonly used to judge the voice activity probability sequence.

[0102] Taking an example, when a single threshold is used and the single threshold = 0.5, the value of the data frame corresponding to the time of the voice presence probability greater than or equal to 0.5 is determined as 1, and the value of the data frame corresponding to the time of the voice presence probability less than 0.5 is determined as 0; after the frame-by-frame comparison, the voice activity signal is output frame by frame in time sequence according to the value of the data frame at each time.

[0103] Taking an example, when a binary threshold is used, i.e. the first threshold value = 0.4 and the second threshold value = 0.6, the value of the data frame corresponding to the time of the voice presence probability greater than or equal to 0.6 is determined as 1, and the value of the data frame corresponding to the time of the voice presence probability less than or equal to 0.4 is determined as 0; for the data frame corresponding to the time of the voice presence probability less than 0.6 and greater than 0.4, the value can be determined with reference to the value of the data frame of the previous frame; after the frame-by-frame comparison, the voice activity signal is output frame by frame in time sequence according to the value of the data frame at each time.

[0104] It is understandable that the preset threshold information can be a single-value threshold; or the preset threshold information includes a first threshold value and a second threshold value; as long as the preset threshold information can be used to implement threshold judgment, this application does not limit the specific form of the preset threshold information.

[0105] Through step S1251 to step S1252, the preset threshold information is used to perform threshold judgment on the voice activity probability sequence to generate a voice activity signal, so as to facilitate subsequent power consumption feedback control based on the obtained voice activity signal.

[0106] Through step S121 to step S125, by using the neural network voice activity detection model to directly perform voice activity detection on the oversampled initial audio sequence, the detection delay is reduced, which is conducive to improving the real-time performance of subsequent voice interaction.

[0107] Specifically, in step S130 , the feedback control signal includes a first control signal of a high level and a second control signal of a low level.

[0108] According to some embodiments of the present application, the level state of the voice activity signal includes: a high level; further explaining step S130, step S130: the power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module, and generates a feedback control signal to control the working mode of the signal acquisition module, including but not limited to steps S131 to S132.

[0109] Step S131: When the level state of the voice activity signal is high, the power consumption feedback control module generates a high-level first control signal in response to the high level.

[0110] Step S132: Control the signal acquisition module to enter the normal mode according to the first control signal.

[0111] According to some embodiments of the present application, the level state of the voice activity signal also includes: low level; further explaining step S130, step S130: the power consumption feedback control module monitors the level state of the voice activity signal transmitted by the neural network voice activity detection module, and generates a feedback control signal to control the working mode of the signal acquisition module, including but not limited to steps S133 to S135.

[0112] Step S133: When the level state of the voice activity signal is low level, the power consumption feedback control module continuously monitors the voice activity signal to obtain a low level duration.

[0113] Step S134: When the low-level duration exceeds a preset time threshold, a low-level second control signal is generated to control the signal acquisition module to enter a low-power consumption mode.

[0114] Step S135: When the low-level duration does not exceed the preset time threshold, the first control signal in the high level is continuously output, and the control signal acquisition module remains in the normal mode.

[0115] Through steps S131 to S135, the valid voice segment is recognized based on the voice activity signal, the control signal acquisition module works in the normal mode to collect the oversampled initial audio sequence, and the normal work of the detection system is ensured; the power consumption feedback control is performed based on the obtained voice activity signal, and the working mode of the signal acquisition module is dynamically adjusted, so that the overall power consumption of the system can be effectively reduced.

[0116] The application does not use the ADC threshold design, but introduces a power consumption feedback control mechanism: the ADC itself has two working modes of "low power consumption" and "normal", when the VAD detects the presence of voice, the ADC is switched from the "low power consumption" mode to the "normal" mode; when the VAD does not detect voice, the ADC is in the "low power consumption" mode. Since the VAD algorithm implemented by the large model itself has a very low false detection rate, the mode switching will not be frequent, and the system power consumption can be effectively improved. The ADC and the VAD algorithm implemented by the large model are integrated in one chip, then the ADC will always generate audio data, and the VAD signal is sent to the subsequent general computing platform for processing, which allows the subsequent algorithm to have strong freedom.

[0117] It should be emphasized that the VAD function is separated from the complex voice processing system in the application, combined with the ADC, integrated in one chip, and forms a general voice front-end chip. In this way, the front-end conversion and the back-end processing are decoupled under the premise of ensuring high accuracy and low power consumption, which greatly improves the universality and ease of use, and users can choose ADCs with different signal-to-noise ratios and CPUs with different computing capabilities, and the two are no longer bound.

[0118] Taking an example, the specific process of power consumption feedback control is described in combination with Figure 8 After obtaining the voice activity signal, first, it is judged whether the level state of the voice activity signal is 1 (high level), if yes, the control signal acquisition module enters the normal working mode; if not, the low-level duration is monitored; it is judged whether the low-level duration is overdue, if yes, the control signal acquisition module enters the low-power mode; if not, the control signal acquisition module enters the normal working mode.

[0119] As Figure 9As shown, when the initial voice signal includes valid voice segments and invalid noise segments; after voice activity detection of the present application, a voice activity signal is obtained; corresponding to the valid voice segment, the voice activity signal is high level; corresponding to the invalid noise segment, the voice activity signal is low level. Based on this, the power consumption feedback control module generates a high-level first control signal in response to the high level of the voice activity signal, and controls the signal collection module to enter the normal mode based on the first control signal. The power consumption feedback control module responds to the low level of the voice activity signal, continuously monitors the voice activity signal to obtain the low level duration; when the low level duration exceeds the preset time threshold, a low-level second control signal is generated to control the signal collection module to enter the low-power mode; when the low level duration does not exceed the preset time threshold, the first control signal of the high level is continuously output, and the signal collection module remains in the normal mode. In this way, the working mode of the signal collection module is flexibly adjusted based on the voice activity signal, ensuring normal processing of the valid voice segment; at the same time, in the case of receiving invalid noise segments, the signal collection module enters the low-power mode, which can effectively reduce the overall power consumption of the system.

[0120] The analog-to-digital converter ADC itself has two working modes: low-power mode and normal mode; when voice is detected (i.e. the VAD signal changes from low level to high level), the ADC is switched from low-power mode to normal mode; when the ADC is in normal mode, the level state (high level or low level, high level is "trigger" state, low level is "untriggered" state) of VAD is continuously monitored, and once VAD is in untriggered state and exceeds the preset time, the ADC returns from "normal" working mode to "low-power" working mode.

[0121] It can be understood that the input of the power consumption feedback control module is the VAD signal (word level), and the output is the equivalent feedback control signal (sentence level), which is used to control the working mode of the analog-to-digital converter ADC in the signal collection module to achieve the purpose of low power consumption of the system. Obviously, due to the diversity of voice, especially the different speaking speeds of speakers, the basic unit of VAD marking is "word" level, and if the "word" level VAD is directly fed back to control the working mode of the ADC, frequent switching will occur. Although the embodiment of the present application allows online switching of the working mode, frequent switching will cause the signal-to-noise ratio of the voice signal to fluctuate greatly, and from the perspective of ensuring signal stability, it is desirable to maintain the signal-to-noise ratio as constant as possible. Frequent switching cannot further significantly improve power consumption, so after comprehensive consideration, the feedback control signal of "sentence" level is fed back to control the working mode of the ADC, which is more balanced in all aspects.

[0122] In an embodiment, as Figure 10As shown, before the voice activity signal detection method shown as steps S110 to S130 is performed, the voice activity signal detection method further includes: first, obtaining oversampling data from a historical database, and designing an initial voice activity detection model to be trained; second, performing model training processing on the designed initial voice activity detection model to be trained based on the oversampling data; third, performing parameter optimization processing on the trained voice activity detection model to be evaluated; and fourth, performing parameter evaluation processing on the optimized voice activity detection model after the parameter optimization processing, the parameter evaluation processing being: determining whether the parameter evaluation passes; if yes, outputting a target neural network training model, and performing model hardware on the neural network training model to complete pre-training; and if no, re-performing the model training processing, the parameter optimization processing, and the parameter evaluation processing until the target neural network training model is obtained and the model hardware is performed to complete the pre-training.

[0123] The present application advances the VAD function implementation position to the audio data conversion stage, and realizes the VAD function through the optimized voice large model before the sound signal is converted into high-speed audio data by the ADC but has not been extracted into low-speed audio data. The voice large model adopted is trained in various complex scenes in advance, and can achieve very high accuracy for human voice.

[0124] Specifically, the model design is generally based on the Transformer architecture, and the number of parameters depends on the target effect of the model. After obtaining the data and the model, the model training is started, and the trained model needs to be further optimized. A suitable loss function is generally selected to measure the performance and error of the model, and an optimization algorithm is used to minimize the loss function, so as to update the model parameters. After the training is completed, the model is hardware in the form of FPGA or ASIC, and a certain number of parameters are generally reserved to fine-tune the model; to adapt to specific scene differences.

[0125] Those skilled in the art can understand that all or some of the steps in the method disclosed above can be implemented as software, firmware, hardware and appropriate combinations thereof. Some or all physical components can be implemented as software executed by a processor such as a central processing unit, a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit such as an application specific integrated circuit.

[0126] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the present application.

Claims

1. A method for detecting a voice activity signal, characterized in that: The invention is applied to a voice activity signal detection system, the system comprising: a neural network voice activity detection module, a signal acquisition module and a power consumption feedback control module in communication with the neural network voice activity detection module; the power consumption feedback control module is also in communication with the signal acquisition module; the neural network voice activity detection module comprises: a pre-trained neural network voice activity detection model; the neural network voice activity detection model comprises: a speech input layer, a feature vector encoding layer, a feature integration layer, a feature vector decoding and recognition layer, and an output layer connected in sequence; the neural network voice activity detection module is a hardware module implemented using an FPGA or ASIC; The method comprises: The neural network voice activity detection module receives at least one oversampled initial audio sequence collected by the signal acquisition module; The neural network voice activity detection model sequentially performs data preprocessing, feature vector encoding, vector fusion, voice activity recognition, and signal conversion based on the oversampled initial audio sequence to obtain a voice activity signal. The voice activity recognition process is used to decode the output multi-head attention vector at the current moment to obtain a voice activity probability sequence. The signal conversion process is used to perform threshold judgment on the voice activity probability sequence, convert the voice activity probability sequence into a voice activity signal based on the obtained judgment result, and output the voice activity signal. The power consumption feedback control module monitors the level of the voice activity signal transmitted by the neural network voice activity detection module and generates a feedback control signal to control the operating mode of the signal acquisition module; the operating modes include: low power consumption mode and normal mode; Wherein, the level state of the voice activity signal further includes: low level; The power consumption feedback control module monitors the level of the voice activity signal transmitted by the neural network voice activity detection module and generates a feedback control signal to control the working mode of the signal acquisition module, including: When the level state of the voice activity signal is: low level, the power consumption feedback control module continuously monitors the voice activity signal to obtain a low level duration; When the low level duration exceeds a preset time threshold, a low-level second control signal is generated to control the signal acquisition module to enter the low power consumption mode; When the low level duration does not exceed the preset time threshold, the high level first control signal is continuously output to control the signal acquisition module to remain in the normal mode.

2. The voice activity signal detection method according to claim 1, wherein: The level state of the voice activity signal includes: high level; The power consumption feedback control module monitors the level of the voice activity signal transmitted by the neural network voice activity detection module and generates a feedback control signal to control the working mode of the signal acquisition module, including: When the level state of the voice activity signal is: high level, the power consumption feedback control module generates a high-level first control signal in response to the high level; The signal acquisition module is controlled to enter a normal mode according to the first control signal.

3. The voice activity signal detection method according to claim 1, wherein: The neural network voice activity detection model sequentially performs data preprocessing, feature vector encoding processing, vector fusion processing, voice activity recognition processing, and signal conversion processing based on the oversampled initial audio sequence to obtain a voice activity signal, including: The oversampled initial audio sequence is subjected to data preprocessing through the speech input layer to obtain an input matrix; wherein the input matrix includes a plurality of elements, and different elements represent feature vectors corresponding to different frame times; Performing feature vector encoding processing according to the input matrix through the feature vector encoding layer to obtain at least one feature vector; Through the feature integration layer, a vector fusion process is performed on at least one feature vector according to a pre-obtained first weight matrix and a preset integration formula to obtain a multi-head attention vector at the current moment; Performing voice activity recognition processing based on the multi-head attention vector at the current moment through the feature vector decoding and recognition layer to obtain a voice activity probability sequence; The output layer performs threshold judgment on the voice activity probability sequence, converts the voice activity probability sequence into a voice activity signal based on the obtained judgment result, and outputs the voice activity signal.

4. The voice activity signal detection method according to claim 3, wherein: The voice activity recognition process is performed based on the multi-head attention vector at the current moment to obtain a voice activity probability sequence, including: Obtain the multi-head attention vector at the current moment; Decode the multi-head attention vector at the current moment to obtain the voice activity probability sequence; the voice activity probability sequence includes: the voice existence probability corresponding to the data frames at different moments; the voice existence probability is used to characterize the probability of the existence of voice in the data frame at the corresponding moment, and the value of the voice existence probability is 0 to 1.

5. The voice activity signal detection method according to claim 4, characterized in that: The performing threshold judgment on the voice activity probability sequence, converting the voice activity probability sequence into a voice activity signal based on the obtained judgment result, and outputting the voice activity signal includes: Obtaining speech presence probabilities corresponding to data frames at different times from the speech activity probability sequence; Comparing the speech presence probability with preset threshold information frame by frame to determine the value of the data frame at the moment corresponding to the speech presence probability; After the frame-by-frame comparison is completed, the voice activity signal is output in time sequence according to the values ​​of the data frames at each moment.

6. The voice activity signal detection method according to claim 5, characterized in that: The comparing the speech presence probability with preset threshold information frame by frame to determine the value of the data frame at the moment corresponding to the speech presence probability includes: When the preset threshold information is a single-value threshold, comparing the speech presence probability with the single-value threshold frame by frame to obtain a first comparison result, and determining a value of the data frame at the time corresponding to the speech presence probability according to the first comparison result; Alternatively, when the preset threshold information includes: a first threshold value and a second threshold value greater than the first threshold value; the probability of speech existence is compared with the first threshold value and the second threshold value frame by frame to obtain a second comparison result, and the value of the data frame at the moment corresponding to the probability of speech existence is determined according to the second comparison result.

7. A voice activity signal detection system, characterized in that: include: a neural network voice activity detection module, a signal acquisition module and a power consumption feedback control module communicatively connected to the neural network voice activity detection module; The power consumption feedback control module is also communicatively connected to the signal acquisition module; wherein the neural network voice activity detection module includes: a pre-trained neural network voice activity detection model; the neural network voice activity detection model includes: a voice input layer, a feature vector encoding layer, a feature integration layer, a feature vector decoding and recognition layer, and an output layer connected in sequence; the neural network voice activity detection module is a hardware module implemented based on FPGA or ASIC; the signal acquisition module, the neural network voice activity detection module, and the power consumption feedback control module cooperate with each other to enable the voice activity signal detection system to implement the voice activity signal detection method according to any one of claims 1 to 6.

8. The voice activity signal detection system according to claim 7, wherein: The neural network voice activity detection model includes: a voice input layer, a feature vector encoding layer, a feature integration layer, a feature vector decoding and recognition layer, and an output layer connected in sequence.

9. The voice activity signal detection system according to claim 7, wherein: The signal acquisition module includes: at least one audio acquisition unit, each of which is used to acquire an oversampled initial audio sequence; and enters a low power consumption mode or a normal mode under the control of the power consumption feedback control module.

Images