Real-time voice activity detection method and system for audio and video conferencing

Abstract

The present invention relates to a real-time voice activity detection method for audio and video conference, comprising obtaining a voice sample to be detected in a real-time conference scene, wherein the voice sample includes pure data and noise data; performing feature extraction on the voice sample to be detected to obtain the LogFbank feature of the voice sample; inputting the LogFbank feature into a neural network model for training to obtain a voice detection model, wherein the neural network model is a combined model; using the voice detection model to perform voice detection on the voice to be detected collected from the real-time conference to obtain the category of the voice to be detected. The present invention extracts the LogFbank feature of the voice for neural network model training and detection, the calculation amount is smaller, and the LogFbank feature carries a larger amount of information, the training effect and the detection effect are both better, and it is more suitable for real-time conference scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of voice detection, and in particular to a real-time voice activity detection method and system for audio and video conferencing. Background Art

[0002] The voice activation detection (VAD) algorithm is an important part of the voice front-end algorithm. Its purpose is to detect the voice from the audio signal obtained by the microphone so that the subsequent algorithm can process it. In real-time conference scenarios, the accuracy of the VAD algorithm has a great impact on the subsequent algorithms and the final sound quality. The traditional VAD method is mainly based on the characteristics of speech for modeling, and has high requirements on the external environment and speech signal-to-noise ratio. It cannot handle some transient noises such as knocking and keyboard sounds.

[0003] In recent years, the VAD method based on neural network has become increasingly popular. It uses the powerful data fitting ability of neural network to detect human voice in complex scenes, and the effect is usually better than traditional algorithms. However, the algorithm itself has a large amount of calculation and parameters, and it cannot strike a good balance between the performance and cost of the algorithm, so it is difficult to directly use it in real-time conference equipment. Summary of the invention

[0004] To this end, the technical problem to be solved by the present invention is to overcome the problems existing in the prior art and propose a real-time voice activity detection method and system for audio and video conferencing. The present invention extracts the LogFbank features of speech for neural network model training and detection, which has a smaller amount of calculation and a larger amount of information carried by the LogFbank features. The training effect and the detection effect are better, and it is more suitable for real-time conference scenarios.

[0005] In order to solve the above technical problems, the present invention provides a real-time voice activity detection method for audio and video conferencing, comprising the following steps:

[0006] Acquire a speech sample to be detected in a real-time conference scene, wherein the speech sample includes pure data and noise data;

[0007] Performing feature extraction on the speech sample to be detected to obtain LogFbank features of the speech sample;

[0008] Inputting the LogFbank features into a neural network model for training to obtain a speech detection model, wherein the neural network model is a combined model;

[0009] The speech detection model is used to perform speech detection on the speech to be detected collected from the real-time conference to obtain the category of the speech to be detected.

[0010] In one embodiment of the present invention, the pure data in the speech sample is collected by a sound collection device in an anechoic room, and the noise data is collected by a sound collection device in different indoor and outdoor scenes.

[0011] In one embodiment of the present invention, after obtaining the voice sample, preprocessing the voice sample includes:

[0012] Use open source tools to filter the clean data and delete the clean data with incorrect sampling point markings;

[0013] Perform data enhancement and noise processing on the filtered clean data;

[0014] Splice silent data at both ends of the processed clean data so that the proportion of human voice and noise in the speech sample is roughly the same;

[0015] The concatenated clean data and the noise data are mixed to obtain a speech training sample.

[0016] In one embodiment of the present invention, feature extraction is performed on the speech sample to be detected to obtain LogFbank features of the speech sample, including:

[0017] Preprocessing the speech sample, wherein the preprocessing includes removing pre-emphasis, framing and windowing;

[0018] Perform fast Fourier transform on the preprocessed speech samples and calculate the energy to obtain its power spectrum;

[0019] Extract the power spectrum into FBank features through a Mel-scaled triangular filter bank;

[0020] Taking the logarithm of the FBank feature, we get the LogFbank feature.

[0021] In one embodiment of the present invention, the speech detection model includes a convolution layer, a maximum pooling layer, a normalization layer, an LSTM layer and a DNN fully connected layer, wherein the convolution layer includes 16 convolution kernels, each of which has a size of 1×8, and the pooling size of the maximum pooling layer is 1×3.

[0022] In one embodiment of the present invention, using the speech detection model to perform speech detection on the speech to be detected collected from the real-time conference to obtain the category of the speech to be detected includes:

[0023] Extract features of the speech to be detected collected from the real-time conference to obtain the LogFbank features of the speech to be detected;

[0024] Inputting the LogFbank feature of the speech to be detected into the speech detection model to obtain the state value of the speech to be detected;

[0025] Compare the state value of the speech to be detected with a preset threshold value. If its state value is greater than the threshold value, it is determined to be a human voice, otherwise it is determined to be noise;

[0026] The speech determined to be human voice is smoothed, and several frames after it are defined as human voice.

[0027] In addition, the present invention also provides a real-time voice activity detection system for audio and video conferencing, comprising:

[0028] A data acquisition module, the data acquisition module is used to acquire a voice sample to be detected in a real-time conference scene, wherein the voice sample includes pure data and noise data;

[0029] A feature extraction module, wherein the feature extraction module is used to extract features of the speech sample to be detected to obtain LogFbank features of the speech sample;

[0030] A model training module, wherein the model training module is used to input the LogFbank features into a neural network model for training to obtain a speech detection model, wherein the neural network model is a combined model;

[0031] A speech detection module is used to use the speech detection model to perform speech detection on the speech to be detected collected from the real-time conference to obtain the category of the speech to be detected.

[0032] In one embodiment of the present invention, the data acquisition module includes:

[0033] A screening unit, the screening unit is used to screen the clean data using an open source tool, and delete the clean data with incorrect sampling point markings;

[0034] A data processing unit, the data processing unit is used to perform data enhancement and noise addition processing on the clean data after screening;

[0035] A splicing unit, the splicing unit is used to splice the silent data at both ends of the processed clean data so that the proportion of human voice and noise in the speech sample is roughly the same;

[0036] A mixing unit is used to mix the concatenated clean data and the noise data to obtain a speech training sample.

[0037] In one embodiment of the present invention, the feature extraction module includes:

[0038] A preprocessing unit, the preprocessing unit is used to preprocess the speech sample, wherein the preprocessing includes removing pre-emphasis, framing and windowing;

[0039] An energy calculation unit, the energy calculation unit is used to perform a fast Fourier transform on the preprocessed speech sample and calculate the energy to obtain its power spectrum;

[0040] An FBank feature extraction unit, the FBank feature extraction unit is used to extract the power spectrum into FBank features through a Mel-scaled triangular filter bank;

[0041] A LogFbank feature acquisition unit is used to obtain a LogFbank feature by taking the logarithm of the FBank feature.

[0042] In one embodiment of the present invention, the speech detection module includes:

[0043] A LogFbank feature extraction unit, wherein the LogFbank feature extraction unit is used to extract features of the speech to be detected collected from the real-time conference to obtain LogFbank features of the speech to be detected;

[0044] A speech detection unit, the speech detection unit is used to input the LogFbank feature of the speech to be detected into the speech detection model to obtain a state value of the speech to be detected;

[0045] A speech classification unit, the speech classification unit is used to compare the state value of the speech to be detected with a preset threshold value, if its state value is greater than the threshold value, it is determined to be a human voice, otherwise it is determined to be noise;

[0046] The speech post-processing unit is used to perform smoothing processing on the speech determined to be human voice, and define several frames thereafter as human voice.

[0047] The above technical solution of the present invention has the following advantages compared with the prior art:

[0048] 1. The present invention extracts the LogFbank features of speech for neural network model training and detection, which has a smaller amount of calculation and a larger amount of information carried by the LogFbank features. The training effect and the detection effect are both better, and the present invention is more suitable for real-time conference scenarios.

[0049] 2. The neural network model of the present invention is a combined model. By rationally constructing the network model, the characteristics of each neural network layer are utilized to the greatest extent to achieve the best performance.

[0050] 3. The speech data of the present invention includes pure data and noise data, and the data volume is larger and wider, so that the trained speech detection model is more robust to various conference scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0052] Figure 1 It is a flow chart of a real-time voice activity detection method for audio and video conferencing according to the present invention.

[0053] Figure 2 The present invention is a hardware structure diagram of a real-time voice activity detection system for audio and video conferencing.

[0054] Among them, the description of the accompanying drawings is as follows: 10, data acquisition module; 20, feature extraction module; 30, model training module; 40, speech detection module. DETAILED DESCRIPTION

[0055] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0056] Embodiment 1

[0057] See also Figure 1 As shown, this embodiment provides a real-time voice activity detection method for audio and video conferencing, comprising the following steps:

[0058] S1: Acquire a voice sample to be detected in a real-time conference scene, wherein the voice sample includes pure data and noise data;

[0059] S2: extracting features of the speech sample to be detected to obtain LogFbank features of the speech sample;

[0060] S3: Inputting the LogFbank feature into a neural network model for training to obtain a speech detection model, wherein the neural network model is a combined model;

[0061] S4: Using the speech detection model to perform speech detection on the speech to be detected collected from the real-time conference, and obtaining the category of the speech to be detected.

[0062] In a real-time voice activity detection method for audio and video conferencing disclosed in the present invention, the pure data in the voice sample is collected in an anechoic room by a sound collection device, and the noise data is collected in different indoor and outdoor scenes by a sound collection device. For example, the pure data in the voice sample is collected in an anechoic room by a microphone, and the noise data is collected in different indoor and outdoor scenes (including streets, parks, and stadiums, etc.) by a sound collection device.

[0063] In a real-time voice activity detection method for audio and video conferencing disclosed in the present invention, the present invention extracts the LogFbank features of speech for neural network model training and detection, which has a smaller amount of calculation and a larger amount of information carried by the LogFbank features. The training effect and the detection effect are both better, and the method is more suitable for real-time conference scenarios.

[0064] In a real-time voice activity detection method for audio and video conferencing disclosed in the present invention, in step S1, after obtaining the voice sample, the voice sample is preprocessed, which includes:

[0065] S1.1: Use open source tools to filter the clean data and delete the clean data with incorrect sampling point labels. For example, use the open source WebRtcVad method to determine the clean data to obtain the label of each sampling point, and then filter out the incorrectly labeled data and delete them.

[0066] S1.2: Perform data enhancement and noise processing on the filtered clean data. Considering that conference environments generally have reverberation, reverberation processing is added to the clean data.

[0067] S1.3: Splice silent data at both ends of the processed clean data to make the proportion of human voice and noise in the speech sample roughly the same. This speech splicing method is more conducive to neural network training;

[0068] S1.4: Mixing the concatenated clean data and the noise data to obtain a speech training sample. For example, mixing the concatenated clean data and the noise data according to different signal-to-noise ratios to obtain a speech training sample.

[0069] In a real-time voice activity detection method for audio and video conferencing disclosed in the present invention, the voice data of the present invention includes pure data and noise data, and the data volume is larger and wider, so that the trained and learned voice detection model is more robust to various conference scenarios.

[0070] In a real-time voice activity detection method for audio and video conferencing disclosed in the present invention, in step S2, feature extraction is performed on the voice sample to be detected to obtain the LogFbank feature of the voice sample, including:

[0071] S2.1: Preprocessing the speech sample, wherein the preprocessing includes removing pre-emphasis, framing and windowing;

[0072] S2.2: Perform fast Fourier transform on the preprocessed speech sample and calculate the energy to obtain its power spectrum;

[0073] S2.3: The power spectrum is extracted into FBank features through a Mel-scale triangular filter bank, where the Mel-scale is a scale that can simulate the law of human ear receiving sound. The human ear is nonlinear when receiving sound and is less sensitive to high frequencies. The conversion function between the Mel-scale and the frequency is as follows:

[0074]

[0075] f(mel)=700*(10 mel / 2595 -1);

[0076] S2.4: Taking the logarithm of the FBank feature to obtain the LogFbank feature. Taking the logarithm can reduce the change of each dimension of the feature, which is beneficial to the training of the neural network.

[0077] In S2.1, the purpose of removing pre-emphasis is: due to the powerful fitting ability of the neural network, the pre-emphasis has no obvious effect here, and removing it can reduce the amount of calculation; the content of framing is: using a frame length of 32ms, a frame shift of 16ms, and 50% overlap of speech; the purpose of windowing is: to prevent frequency leakage, a Hamming window is used for windowing, and the Hamming window function is

[0078] In S2.3, the Mel filter bank is a series of triangular filters, with a response value of 1 at the center frequency point and attenuated to 0 at the center points of the filters on both sides. This scheme takes 40 groups. By applying the Mel filter bank on the energy spectrum, the FBank feature can be extracted.

[0079] In a real-time voice activity detection method for audio and video conferencing disclosed in the present invention, the voice detection model includes a convolution layer, a maximum pooling layer, a normalization layer, an LSTM layer and a DNN fully connected layer, wherein the convolution layer includes 16 convolution kernels, each of which has a size of 1×8, and the pooling size of the maximum pooling layer is 1×3.

[0080] Specifically, the extracted LogFbank features are input to the first layer of the model, i.e., the convolution layer. The convolution layer consists of 16 convolution kernels, each of which has a size of 1×8. The convolution kernel is convolved on the time-frequency axis. This step is to learn the association information between frequency subbands. The PRelu activation function is then used for calculation, followed by the maximum pooling layer with a pooling size of 1×3. The pooled output is then sent to the normalization layer, which normalizes each feature map, which can effectively reduce the occurrence of misjudgments caused by changes in speech amplitude. The output is then sent to the LSTM layer, which can effectively learn the association information between frames, which plays a great role in improving the accuracy of speech detection. Finally, it is sent to the DNN fully connected layer for classification, and the final frame prediction result is output through the sigmoid function.

[0081] Secondly, the loss function for the general binary classification problem is the mean square error (MSE). The disadvantage is that if there are outliers in the training sample, the MSE will give the outliers a higher weight, but it does sacrifice the prediction effect of other normal points, which will eventually lead to a decrease in the overall performance of the model. Therefore, the loss function used in the present invention is the Huber function, which is defined as follows:

[0082]

[0083] The Huber loss contains a hyperparameter δ. The Huber loss reduces the sensitivity to outliers and achieves the function of being differentiable everywhere. Its effect is better than the MSE loss function.

[0084] In a real-time voice activity detection method for audio and video conferencing disclosed in the present invention, the neural network model of the present invention is a combined model. By reasonably constructing the network model, the characteristics of each neural network layer are utilized to the greatest extent to achieve the best performance.

[0085] In a real-time voice activity detection method for audio and video conferencing disclosed in the present invention, in step S4, using the voice detection model to perform voice detection on the voice to be detected collected from the real-time conference to obtain the category of the voice to be detected, including:

[0086] S4.1: extracting features of the speech to be detected collected from the real-time conference to obtain LogFbank features of the speech to be detected;

[0087] S4.2: Input the LogFbank feature of the speech to be detected into the speech detection model to obtain the state value of the speech to be detected;

[0088] S4.3: Compare the state value of the speech to be detected with a preset threshold value. If the state value is greater than the threshold value, it is determined to be a human voice, otherwise it is determined to be noise;

[0089] S4.4: Smoothing the speech determined to be human voice, and defining several frames after it as human voice.

[0090] In S4.4, usually, since the algorithm is processed in real time at the frame level, the result will have a certain amount of misjudgment, so some post-processing is needed to correct the result of the preliminary judgment to reduce misjudgment and make the speech segment continuous. Therefore, in the present invention, for the current frame to be calculated, if its prediction result is human voice, a certain smoothing process is performed, and it is considered that the N frames after it are all human voices. N can be adjusted according to the actual effect. In the present invention, N can be 30.

[0091] In a real-time voice activity detection method for audio and video conferencing disclosed in the present invention, the present invention post-processes the prediction results of the neural network according to the upper and lower frame information of the voice stream, thereby reducing the probability of voice discontinuity and further improving the performance of the model in actual application scenarios.

[0092] Embodiment 2

[0093] A real-time voice activity detection system for audio and video conferencing disclosed in Embodiment 2 of the present invention is introduced below. The real-time voice activity detection system for audio and video conferencing described below and the real-time voice activity detection method for audio and video conferencing described above can refer to each other.

[0094] See also Figure 2 As shown, Embodiment 2 of the present invention discloses a real-time voice activity detection system for audio and video conferencing, including:

[0095] A data acquisition module 10, wherein the data acquisition module 10 is used to acquire a speech sample to be detected in a real-time conference scene, wherein the speech sample includes pure data and noise data;

[0096] A feature extraction module 20, wherein the feature extraction module 20 is used to extract features of the speech sample to be detected to obtain LogFbank features of the speech sample;

[0097] A model training module 30, wherein the model training module 30 is used to input the LogFbank feature into a neural network model for training to obtain a speech detection model, wherein the neural network model is a combined model;

[0098] The speech detection module 40 is used to perform speech detection on the speech to be detected collected from the real-time conference using the speech detection model to obtain the category of the speech to be detected.

[0099] In a real-time voice activity detection system for audio and video conferencing disclosed in the present invention, the data acquisition module 10 includes:

[0100] A screening unit, the screening unit is used to screen the clean data using an open source tool, and delete the clean data with incorrect sampling point markings;

[0101] A data processing unit, the data processing unit is used to perform data enhancement and noise addition processing on the clean data after screening;

[0102] A splicing unit, the splicing unit is used to splice the silent data at both ends of the processed clean data so that the proportion of human voice and noise in the speech sample is roughly the same;

[0103] A mixing unit is used to mix the concatenated clean data and the noise data to obtain a speech training sample.

[0104] In a real-time voice activity detection system for audio and video conferencing disclosed in the present invention, the feature extraction module 20 includes:

[0105] A preprocessing unit, the preprocessing unit is used to preprocess the speech sample, wherein the preprocessing includes removing pre-emphasis, framing and windowing;

[0106] An energy calculation unit, the energy calculation unit is used to perform a fast Fourier transform on the preprocessed speech sample and calculate the energy to obtain its power spectrum;

[0107] An FBank feature extraction unit, the FBank feature extraction unit is used to extract the power spectrum into FBank features through a Mel-scaled triangular filter bank;

[0108] A LogFbank feature acquisition unit is used to obtain a LogFbank feature by taking the logarithm of the FBank feature.

[0109] In a real-time voice activity detection system for audio and video conferencing disclosed in the present invention, the voice detection module 40 includes:

[0110] A LogFbank feature extraction unit, wherein the LogFbank feature extraction unit is used to extract features of the speech to be detected collected from the real-time conference to obtain LogFbank features of the speech to be detected;

[0111] A speech detection unit, the speech detection unit is used to input the LogFbank feature of the speech to be detected into the speech detection model to obtain a state value of the speech to be detected;

[0112] A speech classification unit, the speech classification unit is used to compare the state value of the speech to be detected with a preset threshold value, if its state value is greater than the threshold value, it is determined to be a human voice, otherwise it is determined to be noise;

[0113] The speech post-processing unit is used to perform smoothing processing on the speech determined to be human voice, and define several frames thereafter as human voice.

[0114] The real-time voice activity detection system for audio and video conferencing of this embodiment is used to implement the aforementioned real-time voice activity detection method for audio and video conferencing. Therefore, the specific implementation method of the system can be seen in the embodiment part of the real-time voice activity detection method for audio and video conferencing in the previous text. Therefore, its specific implementation method can refer to the description of the corresponding various partial embodiments and will not be introduced in detail here.

[0115] In addition, since the real-time voice activity detection system for audio and video conferencing of this embodiment is used to implement the aforementioned real-time voice activity detection method for audio and video conferencing, its function corresponds to that of the aforementioned method and will not be described in detail here.

[0116] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0117] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0120] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A real-time voice activity detection method for audio and video conferencing, characterized in that: The following steps are involved: Acquire a speech sample to be detected in a real-time conference scene, wherein the speech sample includes pure data and noise data; Performing feature extraction on the speech sample to be detected to obtain LogFbank features of the speech sample; Inputting the LogFbank feature into a neural network model for training to obtain a speech detection model, wherein the neural network model is a combined model, and the neural network model includes a convolutional layer, a maximum pooling layer, a normalization layer, an LSTM layer, and a DNN fully connected layer; The speech detection model is used to perform speech detection on the speech to be detected collected from the real-time conference to obtain the category of the speech to be detected.

2. The real-time voice activity detection method for audio and video conferencing according to claim 1, characterized in that: The pure data in the speech sample is collected in an anechoic room by a sound collection device, and the noise data is collected in different indoor and outdoor scenes by a sound collection device.

3. The real-time voice activity detection method for audio and video conferencing according to claim 1, characterized in that: After obtaining the voice sample, preprocessing the voice sample includes: Use open source tools to filter the clean data and delete the clean data with incorrect sampling point markings; Perform data enhancement and noise processing on the filtered clean data; Splice silent data at both ends of the processed clean data so that the proportion of human voice and noise in the speech sample is roughly the same; The concatenated clean data and the noise data are mixed to obtain a speech training sample.

4. The real-time voice activity detection method for audio and video conferencing according to claim 1, characterized in that: Performing feature extraction on the speech sample to be detected to obtain the LogFbank feature of the speech sample includes: Preprocessing the speech sample, wherein the preprocessing includes removing pre-emphasis, framing and windowing; Perform fast Fourier transform on the preprocessed speech samples and calculate the energy to obtain its power spectrum; Extract the power spectrum into FBank features through a Mel-scaled triangular filter bank; Taking the logarithm of the FBank feature, we get the LogFbank feature.

5. The real-time voice activity detection method for audio and video conferencing according to claim 1, characterized in that: The convolution layer includes 16 convolution kernels, each of which has a size of 1×8, and the pooling size of the maximum pooling layer is 1×3.

6. The real-time voice activity detection method for audio and video conferencing according to claim 1, characterized in that: Using the speech detection model to perform speech detection on the speech to be detected collected from the real-time conference to obtain the category of the speech to be detected, including: Extract features of the speech to be detected collected from the real-time conference to obtain the LogFbank features of the speech to be detected; Inputting the LogFbank feature of the speech to be detected into the speech detection model to obtain the state value of the speech to be detected; Compare the state value of the speech to be detected with a preset threshold value. If its state value is greater than the threshold value, it is determined to be a human voice, otherwise it is determined to be noise; The speech determined to be human voice is smoothed, and several frames after it are defined as human voice.

7. A real-time voice activity detection system for audio and video conferencing, characterized in that: include: A data acquisition module, the data acquisition module is used to acquire a voice sample to be detected in a real-time conference scene, wherein the voice sample includes pure data and noise data; A feature extraction module, wherein the feature extraction module is used to extract features of the speech sample to be detected to obtain LogFbank features of the speech sample; A model training module, wherein the model training module is used to input the LogFbank feature into a neural network model for training to obtain a speech detection model, wherein the neural network model is a combined model, and the neural network model includes a convolutional layer, a maximum pooling layer, a normalization layer, an LSTM layer, and a DNN fully connected layer; A speech detection module is used to use the speech detection model to perform speech detection on the speech to be detected collected from the real-time conference to obtain the category of the speech to be detected.

8. The real-time voice activity detection system for audio and video conferencing according to claim 7, characterized in that: The data acquisition module comprises: A screening unit, the screening unit is used to screen the clean data using an open source tool, and delete the clean data with incorrect sampling point markings; A data processing unit, the data processing unit is used to perform data enhancement and noise addition processing on the clean data after screening; A splicing unit, the splicing unit is used to splice the silent data at both ends of the processed clean data so that the proportion of human voice and noise in the speech sample is roughly the same; A mixing unit is used to mix the concatenated clean data and the noise data to obtain a speech training sample.

9. The real-time voice activity detection system for audio and video conferencing according to claim 7, characterized in that: The feature extraction module comprises: A preprocessing unit, the preprocessing unit is used to preprocess the speech sample, wherein the preprocessing includes removing pre-emphasis, framing and windowing; An energy calculation unit, the energy calculation unit is used to perform a fast Fourier transform on the preprocessed speech sample and calculate the energy to obtain its power spectrum; An FBank feature extraction unit, the FBank feature extraction unit is used to extract the power spectrum into FBank features through a Mel-scaled triangular filter bank; A LogFbank feature acquisition unit is used to obtain a LogFbank feature by taking the logarithm of the FBank feature.

10. The real-time voice activity detection system for audio and video conferencing according to claim 7, characterized in that: The speech detection module comprises: A LogFbank feature extraction unit, wherein the LogFbank feature extraction unit is used to extract features of the speech to be detected collected from the real-time conference to obtain LogFbank features of the speech to be detected; A speech detection unit, the speech detection unit is used to input the LogFbank feature of the speech to be detected into the speech detection model to obtain a state value of the speech to be detected; A speech classification unit, the speech classification unit is used to compare the state value of the speech to be detected with a preset threshold value, if its state value is greater than the threshold value, it is determined to be a human voice, otherwise it is determined to be noise; The speech post-processing unit is used to perform smoothing processing on the speech determined to be human voice, and define several frames thereafter as human voice.

Images