A method and device for detecting human voice in a multi-sound-source fusion scene

Abstract

The application discloses a method and device for detecting human voice in a multi-sound-source fusion scene, and provides the method for detecting human voice in the multi-sound-source fusion scene, which uses a first speech frame sample and a second speech frame sample as positive samples and negative samples respectively, generates MFCC feature areas and a spectrogram by using the positive samples and the negative samples, performs human voice MFCC feature recognition training on a deep learning network unit by using the generated spectrogram, and performs human voice detection training on a recurrent neural network unit by using human voice MFCC features output by the deep learning network unit, so as to improve the human voice detection capability of the neural network in a multi-sound-source environment, construct a human voice detection model, and detect human voice in a multi-sound-source sound signal by using the constructed human voice detection model, thereby obtaining a human voice detection result of the multi-sound-source sound signal, and solving the technical problem that the existing multi-sound-source human voice detection cannot accurately detect human voice.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to a method and apparatus for human voice detection in multi-sound source fusion scenarios. Background Technology

[0002] With the rapid development of artificial intelligence technology in recent years, various interaction methods have continued to evolve. Among them, intelligent voice interaction plays a crucial role. Currently, many challenges remain to be solved in the process of intelligent voice interaction, such as sound source recognition, sound source localization, and sound source detection. Among sound source detection, one of the most important directions is human voice detection. Specifically, this refers to the technology for detecting human voices in situations involving multiple mixed sound sources.

[0003] In real life, the greatest need for human voice detection technology is in large-scale scenarios involving multi-source sound fusion (such as cocktail parties, which include various sound sources such as human voices, animal sounds, and machine sounds). In such situations, sound source detection is affected by various factors, making it difficult to accurately detect human voices. Summary of the Invention

[0004] This application provides a method and apparatus for human voice detection in multi-source fusion scenarios, which solves the technical problem that existing sound source detection is easily affected by various factors and is difficult to accurately detect human voices.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a method for human voice detection in a multi-source fusion scenario, comprising:

[0006] Acquire a first speech frame sample and a second speech frame sample, wherein the first speech frame sample is a multi-source speech frame signal sample containing human voice, and the second speech frame sample is a multi-source speech frame signal sample not containing human voice.

[0007] Extract the MFCC features of the first speech frame sample and the second speech frame sample, and then generate the corresponding spectrogram based on the MFCC features;

[0008] Using the spectrograms of the first and second speech frame samples, the deep learning network units are trained for human voice MFCC feature recognition.

[0009] The human voice MFCC features output by the deep learning network unit are used to train the recurrent neural network unit for human voice detection.

[0010] A human voice detection model is constructed based on the trained deep learning network units and recurrent neural network units;

[0011] Acquire the multi-source sound signals to be detected;

[0012] The multi-source sound signals are subjected to windowing and framing processing to obtain several speech frame signals;

[0013] Extract the MFCC features of the speech frame signal, and then generate the spectrogram of the speech frame signal based on the MFCC features;

[0014] The spectrogram is used as the input to the voice detection model, and the voice detection result of the multi-source sound signal is obtained through the operation of the voice detection model.

[0015] Preferably, the step of extracting the MFCC features of the speech frame signal and then generating the spectrogram of the speech frame signal based on the MFCC features specifically includes:

[0016] The speech frame signal is frequency domain transformed using Fourier transform to obtain the speech frame spectrum.

[0017] The speech frame spectrum is processed by Mel filtering through a preset Mel triangular filter bank to obtain several filter bank energies. Then, the filter bank energies are logarithmically transformed to obtain the Fbank features of the speech frame signal.

[0018] The Fbank features are converted into MFCC features using the IDFT transformation method.

[0019] The spectrogram of the speech frame signal is generated based on the MFCC features.

[0020] Preferably, after acquiring the multi-source sound signal to be detected, the method further includes:

[0021] The high-frequency portion of the multi-source sound signal is preprocessed with signal emphasis.

[0022] Preferably, it further includes:

[0023] Based on the human voice detection results of each of the aforementioned voice frame signals, a first voice frame signal is determined, wherein the first voice frame signal is the voice frame signal in which human voice is detected.

[0024] Based on the time period corresponding to each of the first speech frame signals in the multi-source sound signal, the time period corresponding to the non-first speech frame signals in the multi-source sound signal is filtered to obtain the first sound signal.

[0025] The first audio signal is subjected to noise reduction and voice enhancement processing respectively, and then the voice data is extracted from the processed first audio signal.

[0026] Preferably, the recurrent neural network unit is specifically an LSTM neural network unit.

[0027] Meanwhile, a second aspect of this application also provides a human voice detection device for multi-source fusion scenarios, comprising:

[0028] The speech sample acquisition unit is used to acquire a first speech frame sample and a second speech frame sample, wherein the first speech frame sample is a multi-source speech frame sample containing human voice, and the second speech sample is a spectrogram of a multi-source speech frame that does not contain human voice.

[0029] The speech sample processing unit is used to extract MFCC features from the first speech frame sample and the second speech frame sample, and then generate corresponding spectrograms based on the MFCC features.

[0030] The first training unit is used to train the deep learning network unit for human voice MFCC feature recognition using the first speech frame sample and the second speech frame sample.

[0031] The second training unit is used to train the recurrent neural network unit for human voice detection using the human voice MFCC features output by the deep learning network unit.

[0032] The voice detection model building unit is used to build a voice detection model based on the trained deep learning network unit and recurrent neural network unit.

[0033] The signal acquisition unit is used to acquire the multi-source sound signals to be detected.

[0034] The frame segmentation processing unit is used to perform windowing and frame segmentation processing on the multi-source sound signals to obtain several speech frame signals.

[0035] The spectrogram generation unit is used to extract the MFCC features of the speech frame signal and then generate the spectrogram of the speech frame signal based on the MFCC features.

[0036] The voice detection processing unit is used to take the spectrogram as the input of the voice detection model, so as to obtain the voice detection result of the multi-source sound signal through the operation of the voice detection model.

[0037] Preferably, the spectrogram generation unit is specifically used for:

[0038] The speech frame signal is frequency domain transformed using Fourier transform to obtain the speech frame spectrum.

[0039] The speech frame spectrum is processed by Mel filtering through a preset Mel triangular filter bank to obtain several filter bank energies. Then, the filter bank energies are logarithmically transformed to obtain the Fbank features of the speech frame signal.

[0040] The Fbank features are converted into MFCC features using the IDFT transformation method.

[0041] The spectrogram of the speech frame signal is generated based on the MFCC features.

[0042] Preferably, it further includes:

[0043] The preprocessing unit is used to perform signal emphasis preprocessing on the high-frequency components of the multi-source sound signal.

[0044] Preferably, it further includes:

[0045] The first speech frame signal determination unit is used to determine the first speech frame signal based on the human voice detection results of each of the speech frame signals, wherein the first speech frame signal is the speech frame signal in which human voice is detected.

[0046] The signal filtering unit is used to filter the time periods that are not corresponding to the first speech frame signals in the multi-source sound signal based on the time periods corresponding to each of the first speech frame signals in the multi-source sound signal, so as to obtain the first sound signal.

[0047] The human voice data extraction unit is used to perform noise reduction processing and human voice enhancement processing on the first sound signal respectively, and then extract human voice data from the processed first sound signal.

[0048] Preferably, the recurrent neural network unit is specifically an LSTM neural network unit.

[0049] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0050] The human voice detection method for multi-source fusion scenarios provided in this application utilizes a first speech frame sample containing human voice and a second speech frame sample not containing human voice as positive and negative samples, respectively. MFCC feature regions and spectrograms are generated using these positive and negative samples. The generated spectrograms are then used to train a deep learning network unit for human voice MFCC feature recognition. Furthermore, the human voice MFCC features output by the deep learning network unit are used to train a recurrent neural network unit for human voice detection, thereby improving the neural network's ability to detect human voices in multi-source environments. Based on the trained deep learning network unit and recurrent neural network unit, a human voice detection model is constructed. This model is then used to detect human voices in multi-source sound signals, thus obtaining the human voice detection results for multi-source sound signals. This solves the technical problem that existing sound source detection methods are easily affected by various factors and struggle to accurately detect human voices. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating an embodiment of a voice detection method for a multi-source fusion scenario provided in this application.

[0053] Figure 2 This is a schematic diagram of the DNN network architecture in a voice detection method for a multi-source fusion scenario provided in this application.

[0054] Figure 3 This is a schematic diagram of the architecture of the LSTM network in the human voice detection method for a multi-source fusion scenario provided in this application.

[0055] Figure 4 This is a schematic diagram illustrating the logical relationship between the two training stages in a voice detection method for a multi-source fusion scenario provided in this application.

[0056] Figure 5 This is a flowchart illustrating another embodiment of a voice detection method for a multi-source fusion scenario provided in this application.

[0057] Figure 6 This is a schematic diagram of one embodiment of a voice detection device for a multi-source fusion scenario provided in this application. Detailed Implementation

[0058] This application provides a method and apparatus for human voice detection in a multi-sound source fusion scenario, which solves the technical problem that existing sound source detection is easily affected by various factors and is difficult to accurately detect human voices.

[0059] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] Please see Figure 1 The first embodiment of this application provides a method for human voice detection in a multi-source fusion scenario, comprising:

[0061] Step S1: Obtain the first speech frame sample and the second speech frame sample.

[0062] The first speech frame sample is a multi-source speech frame signal sample containing human voice, and the second speech frame sample is a multi-source speech frame signal sample not containing human voice. The speech frame mentioned in this embodiment specifically refers to the speech segment after the source speech data is framed.

[0063] It should be noted that, firstly, it is necessary to collect audio frame sample data. This solution defines a large-scale scene with multiple sound sources as follows: an open scene containing sound sources from people, animals, machines, or other objects. These scenes are very common in reality, such as many amusement park scenes, animal-themed scenes, etc. A well-known example is the cocktail scene. Deep learning training requires a balanced number of positive and negative samples. The cocktail scene mentioned above is a positive sample, and negative samples are also needed. Negative samples are easy to select. In this embodiment, the focus is on human voices, so any large-scale scene sound sources without human voices can be used as negative samples for training.

[0064] Step S2: Extract the MFCC features of the first speech frame sample and the second speech frame sample, and then generate the corresponding spectrogram based on the MFCC features.

[0065] Then, based on the collected first and second speech frame samples, the MFCC (Mel-frequency cepstral coefficients) features of the above speech frame samples are extracted, and the corresponding spectrogram is generated based on the MFCC features. It can be understood that the extracted MFCC features are included in the generated spectrogram.

[0066] Step S3: Use the spectrograms of the first and second speech frame samples to train the deep learning network unit for human voice MFCC feature recognition.

[0067] Step S4: Use the human voice MFCC features output by the deep learning network unit to train the recurrent neural network unit for human voice detection.

[0068] Step S5: Based on the trained deep learning network units and recurrent neural network units, construct a human voice detection model.

[0069] It should be noted that steps S3 to S5 in this embodiment constitute the training process of the voice detection model, such as... Figure 4 As shown, the overall approach can be divided into two stages. Stage one is used to detect the presence of human voices. Stage two is based on the positive samples detected in stage one (samples predicted to contain human voices) and is specifically used to label the frame segments containing human voices.

[0070] Phase 1 Deep Learning Network Training:

[0071] The first step is feature extraction. Input: a 224*224 spectrogram; Output: 1000-dimensional output features. Based on the VGG19 network, these 1000-dimensional features are input into a custom DNN network for prediction and labeling. The DNN structure diagram mentioned in this embodiment is shown below. Figure 2 As shown, the main network functions include:

[0072] Activation function: sigmoid function

[0073] Loss function (L): Cross-entropy,

[0074] After Phase 1, positive and negative prediction labels for all spectrograms can be obtained. The MFCC features (39 dimensions) corresponding to the positive samples are selected for the second phase of training, and the frame segments containing human voices are labeled.

[0075] Then, human voice prediction is performed. If the prediction includes human voice, the next step is to detect the location of the human voice. This step uses a recurrent neural network, preferably the classic Long Short-Term Memory (LSTM) network from the recurrent neural network (RNN) family. Its network structure is as follows: Figure 3 As shown, segments containing human voices are labeled by inputting MFCC features. The input consists of 39-dimensional MFCC features for each speech frame segment. The outputs include: Output 1: Human voices are present; Output 0: No human voices are present.

[0076] Overall loss function: Loss = α1*y 1_loss +α2*y 2_loss α1+α2=1, and α1>α2 is set. In this embodiment, α1=0.3 and α2=0.7 are set, that is, the penalty for the first-stage detection error is considered to be greater than that for the second stage.

[0077] Based on this loss function, the backpropagation algorithm is used for training and parameter tuning, and finally a well-trained human voice detection model is obtained.

[0078] Step S6: Acquire the multi-source sound signals to be detected.

[0079] Step S7: Window and frame the multi-source sound signals to obtain several speech frame signals.

[0080] It should be noted that the reason for framing in this embodiment is that speech signals are non-stationary signals, and their statistical properties change over time. Taking Chinese as an example, a sentence contains many initials and finals, and the pronunciation characteristics of different pinyin are obviously different. However, speech signals also have short-term stationary properties. For example, an initial or final in Chinese often only lasts for tens to hundreds of milliseconds. Within this single unit of pronunciation, the speech signal exhibits obvious stability and regularity. Therefore, when performing speech recognition, for a sentence, the recognition process is also performed on a smaller unit of pronunciation (phoneme, character, byte). Thus, a sliding window is used to extract short-time segments. In the time domain, the framing process involves multiplying the original signal by a window function, y[n] = x[n]w[n], where w[n] is called the window function. This scheme uses a Hamming window, the expression of which is:

[0081]

[0082] Where L is the number of sampling points corresponding to a frame length of 25ms, which is 400 here, and n is the sampling point number corresponding to a segment of speech signal. The continuous-time sound wave signal (.wav) is stored as a discrete digital sequence signal in the computer.

[0083] Step S8: Extract the MFCC features of the speech frame signal, and then generate the spectrogram of the speech frame signal based on the MFCC features.

[0084] Step S9: Use the spectrogram as the input to the voice detection model to obtain the voice detection results of multi-source sound signals through the operation of the voice detection model.

[0085] Based on the voice detection model constructed in steps S1 to S5, when actually performing voice detection of multi-source sound signals, it is only necessary to acquire the multi-source sound signals to be detected, and then perform processing steps such as framing, MFCC feature extraction, and spectrogram generation on the multi-source sound signals in sequence. Then, the obtained spectrogram is input into the above-mentioned voice detection model so that the voice detection result of the multi-source sound signals can be obtained through the operation of the voice detection model.

[0086] The above content is a detailed description of the first embodiment of a voice detection method for a multi-source fusion scene provided by this application. The following content is a detailed description of the second embodiment of a voice detection method for a multi-source fusion scene provided by this application based on the content of the first embodiment above.

[0087] Please refer to Figure 5 The second embodiment of this application provides a method for human voice detection in a multi-source fusion scenario, which, based on the content provided in the first embodiment, further includes the following:

[0088] Furthermore, step S8 mentioned in the first embodiment specifically includes:

[0089] The speech frame spectrum is obtained by performing frequency domain transformation on the speech frame signal using Fourier transform processing.

[0090] It should be noted that after the previous step of framing the speech frames, the time domain is transformed to the frequency domain, and the modulus of the Discrete Fourier Transform (DFT) coefficients is taken to obtain the spectral features. Fourier analysis is the most fundamental method of signal analysis, and the Fourier transform is the core of Fourier analysis. It transforms the signal from the time domain to the frequency domain, thereby allowing the study of the signal's spectral structure and variation patterns.

[0091] The speech frame spectrum is processed by Mel filtering using a preset Mel triangular filter bank to obtain the energy of several filter banks. Then, the filter bank energy is logarithmically transformed to obtain the Fbank feature of the speech frame signal.

[0092] The Fbank features are transformed into MFCC features using the IDFT transformation method.

[0093] It should be noted that while the DFT obtains the energy of the signal in each frequency band, the human ear's perception of frequency is not evenly spaced, but approximates a logarithmic function.

[0094] • Convert linear frequencies (0–8 kHz) in speech to Mel frequencies. The conversion relationship between Mel frequencies and linear frequencies is as follows:

[0095]

[0096] Mel-triangle filter bank: The coefficients of each filter are determined based on the start frequency, intermediate frequency, and cutoff frequency (m is the number of filters).

[0097]

[0098] After passing through the Mel filter, the energies of m filter banks are obtained. A log operation is then performed to obtain the Fbank features. Following these steps, further operations such as IDFT transformation are performed on the Fbank features to obtain the MFCC features.

[0099] Spectrograms of speech frame signals generated based on MFCC features.

[0100] Furthermore, after S6, it also includes:

[0101] Step S61: Perform signal emphasis preprocessing on the high-frequency part of the multi-source sound signal.

[0102] It should be noted that the reason for pre-emphasis in this embodiment is to increase the energy of the high-frequency part of the signal. High-frequency signals attenuate quickly during transmission, but the high-frequency part contains many features that are beneficial to speech recognition. Therefore, in the feature extraction part, it is necessary to increase the energy of the high-frequency part.

[0103] The pre-emphasis filter is a first-order high-pass filter. Given a time-domain input signal x[n], the pre-emphasized signal is:

[0104] y[n] = x[n] - αx[n-1]

[0105] Where 0.9≤α≤1.0, pre-emphasis is a high-pass filtering process; therefore, low-frequency signals (signals with slow signal transformation in the time domain) will be suppressed. From the formula, we can know that:

[0106] (1) If signal x is a low-frequency signal (changes slowly), then the values ​​of x[n] and x[n-1] should be very close. When α is close to 1, x[n]-αx[n-1] is close to 0, and the amplitude of this signal will be greatly suppressed.

[0107] (2) If x is a high-frequency signal (changing rapidly), then the values ​​of x[n] and x[n-1] will differ greatly, and the value of x[n]-αx[n-1] will not approach 0. The amplitude of this signal can still be maintained, and it can pass through this filter.

[0108] The larger the α value, the more significant the suppression of low-frequency signals.

[0109] Furthermore, step S9 may also include:

[0110] Step S10: Determine the first speech frame signal based on the human voice detection results of each speech frame signal, wherein the first speech frame signal is the speech frame signal in which human voice is detected.

[0111] Step S11: Based on the time periods corresponding to each first speech frame signal in the multi-source sound signal, filter the time periods corresponding to non-first speech frame signals in the multi-source sound signal to obtain the first sound signal.

[0112] Step S12: Perform noise reduction and voice enhancement processing on the first audio signal respectively, and then extract the voice data from the processed first audio signal.

[0113] It should be noted that, based on the above output, a filter can be set to filter the original audio according to the output result, and speech segments containing human voices can be obtained. On this basis, noise reduction and human voice enhancement can be performed on the processed speech segments to extract human voice data of a certain quality for use.

[0114] The above content is a detailed description of the second embodiment of the human voice detection method for a multi-source fusion scene provided by this application. The following content is a detailed description of an embodiment of the human voice detection device for a multi-source fusion scene provided by this application.

[0115] Please see Figure 6 The third embodiment of this application also provides a human voice detection device for a multi-sound source fusion scenario, including:

[0116] The speech sample acquisition unit C1 is used to acquire a first speech frame sample and a second speech frame sample, wherein the first speech frame sample is a multi-source speech frame sample containing human voice, and the second speech frame sample is a spectrogram of a multi-source speech frame that does not contain human voice.

[0117] The speech sample processing unit C2 is used to extract the MFCC features of the first speech frame sample and the second speech frame sample, and then generate the corresponding spectrogram based on the MFCC features.

[0118] The first training unit C3 is used to train the deep learning network unit for human voice MFCC feature recognition using the first speech frame sample and the second speech frame sample.

[0119] The second training unit C4 is used to train the recurrent neural network unit for human voice detection by utilizing the human voice MFCC features output by the deep learning network unit.

[0120] The voice detection model building unit C5 is used to build a voice detection model based on the trained deep learning network unit and recurrent neural network unit.

[0121] The signal acquisition unit C6 is used to acquire the multi-source sound signals to be detected.

[0122] The frame segmentation processing unit C7 is used to perform windowing and frame segmentation processing on multi-source sound signals to obtain several speech frame signals.

[0123] The spectrogram generation unit C8 is used to extract the MFCC features of the speech frame signal and then generate the spectrogram of the speech frame signal based on the MFCC features.

[0124] The voice detection processing unit C9 is used to take the spectrogram as the input of the voice detection model, so as to obtain the voice detection results of multi-source sound signals through the operation of the voice detection model.

[0125] Furthermore, the spectrogram generation unit C8 is specifically used for:

[0126] The speech frame spectrum is obtained by performing frequency domain transformation on the speech frame signal using Fourier transform processing.

[0127] The speech frame spectrum is processed by Mel filtering using a preset Mel triangular filter bank to obtain the energy of several filter banks. Then, the filter bank energy is logarithmically transformed to obtain the Fbank feature of the speech frame signal.

[0128] The Fbank features are transformed into MFCC features using the IDFT transformation method.

[0129] Spectrograms of speech frame signals generated based on MFCC features.

[0130] Furthermore, it also includes:

[0131] The preprocessing unit C61 is used to perform signal emphasis preprocessing on the high-frequency part of the multi-source sound signal.

[0132] Furthermore, it also includes:

[0133] The first speech frame signal determination unit C10 is used to determine the first speech frame signal based on the human voice detection results of each speech frame signal, wherein the first speech frame signal is the speech frame signal in which human voice is detected.

[0134] The signal filtering unit C11 is used to filter the time periods that are not corresponding to the first speech frame signals in the multi-source sound signals based on the time periods corresponding to each first speech frame signal in the multi-source sound signals, so as to obtain the first sound signal.

[0135] The human voice data extraction unit C12 is used to perform noise reduction processing and human voice enhancement processing on the first sound signal respectively, and then extract human voice data from the processed first sound signal.

[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the terminals, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0137] In the several embodiments provided in this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0138] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0139] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0140] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0141] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for human voice detection in a multi-source fusion scene, characterized in that, include: Obtain a first speech frame sample and a second speech frame sample, wherein the first speech frame sample is a multi-source speech frame sample containing human voice, and the second speech frame sample is a spectrogram of a multi-source speech frame that does not contain human voice. Extract the MFCC features of the first speech frame sample and the second speech frame sample, and then generate the corresponding spectrogram based on the MFCC features; Using the first speech frame sample and the second speech frame sample, the deep learning network unit is trained for human voice MFCC feature recognition, wherein the deep learning network is specifically a DNN network based on the VGG19 network architecture. The human voice MFCC features output by the deep learning network unit are used to train the recurrent neural network unit for human voice detection. The recurrent neural network unit is specifically an LSTM neural network unit. A human voice detection model is constructed based on the trained deep learning network units and recurrent neural network units; Acquire the multi-source sound signals to be detected; The multi-source sound signals are subjected to windowing and framing processing to obtain several speech frame signals; The speech frame signal is frequency domain transformed using Fourier transform to obtain the speech frame spectrum. The speech frame spectrum is processed by Mel filtering through a preset Mel triangular filter bank to obtain several filter bank energies. Then, the filter bank energies are logarithmically transformed to obtain the Fbank features of the speech frame signal. The Fbank features are converted into MFCC features using the IDFT transformation method. The spectrogram of the speech frame signal is generated based on the MFCC features; The spectrogram is used as the input to the voice detection model, and the voice detection result of the multi-source sound signal is obtained through the operation of the voice detection model.

2. The method for human voice detection in a multi-source fusion scene according to claim 1, characterized in that, After acquiring the multi-source sound signal to be detected, the process also includes: The high-frequency portion of the multi-source sound signal is preprocessed with signal emphasis.

3. The method for human voice detection in a multi-source fusion scene according to claim 1, characterized in that, Also includes: Based on the human voice detection results of each of the aforementioned voice frame signals, a first voice frame signal is determined, wherein the first voice frame signal is the voice frame signal in which human voice is detected. Based on the time period corresponding to each of the first speech frame signals in the multi-source sound signal, the time period corresponding to the non-first speech frame signals in the multi-source sound signal is filtered to obtain the first sound signal. The first audio signal is subjected to noise reduction and voice enhancement processing respectively, and then the voice data is extracted from the processed first audio signal.

4. A human voice detection device for multi-source fusion scenarios, characterized in that, include: The speech sample acquisition unit is used to acquire a first speech frame sample and a second speech frame sample, wherein the first speech frame sample is a multi-source speech frame sample containing human voice, and the second speech sample is a spectrogram of a multi-source speech frame that does not contain human voice. The speech sample processing unit is used to extract MFCC features from the first speech frame sample and the second speech frame sample, and then generate corresponding spectrograms based on the MFCC features. The first training unit is used to train the deep learning network unit for human voice MFCC feature recognition using the first speech frame sample and the second speech frame sample. The second training unit is used to train the recurrent neural network unit for human voice detection using the human voice MFCC features output by the deep learning network unit. The voice detection model building unit is used to build a voice detection model based on the trained deep learning network unit and recurrent neural network unit. The signal acquisition unit is used to acquire the multi-source sound signals to be detected. The frame segmentation processing unit is used to perform windowing and frame segmentation processing on the multi-source sound signals to obtain several speech frame signals. The spectrogram generation unit is used to perform frequency domain transformation on the speech frame signal using Fourier transform processing to obtain the speech frame spectrum; to perform Mel filtering processing on the speech frame spectrum using a preset Mel triangular filter bank to obtain several filter bank energies; to perform logarithmic transformation on the filter bank energies to obtain the Fbank features of the speech frame signal; to convert the Fbank features into MFCC features using IDFT transform processing; and to generate the spectrogram of the speech frame signal based on the MFCC features. The voice detection processing unit is used to take the spectrogram as the input of the voice detection model, so as to obtain the voice detection result of the multi-source sound signal through the operation of the voice detection model.

5. A human voice detection device for a multi-source fusion scene according to claim 4, characterized in that, Also includes: The preprocessing unit is used to perform signal emphasis preprocessing on the high-frequency components of the multi-source sound signal.

6. The human voice detection device for a multi-source fusion scene according to claim 4, characterized in that, Also includes: The first speech frame signal determination unit is used to determine the first speech frame signal based on the human voice detection results of each of the speech frame signals, wherein the first speech frame signal is the speech frame signal in which human voice is detected. The signal filtering unit is used to filter the time periods that are not corresponding to the first speech frame signals in the multi-source sound signal based on the time periods corresponding to each of the first speech frame signals in the multi-source sound signal, so as to obtain the first sound signal. The human voice data extraction unit is used to perform noise reduction processing and human voice enhancement processing on the first sound signal respectively, and then extract human voice data from the processed first sound signal.

Images