Training method and device of fundamental frequency extraction model, fundamental frequency extraction method and device
By performing noise-added processing on clean speech and extracting features, using short-time Fourier transform and downsampling to calculate harmonic product spectrum, the fundamental frequency extraction model is solved, and the problem of low training efficiency of fundamental frequency extraction model in the existing technology is solved. Achieve more efficient fundamental frequency extraction and real-time performance.
Patent Information
- Application Number
- CN202111407245.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-24
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-11-24
AI Technical Summary
In the prior art, the training efficiency of the fundamental frequency extraction model is low, the traditional signal method performs poorly in noisy environments, and the calculation amount of methods based on neural network models is large, making it difficult to meet the real-time requirements.
By extracting the fundamental frequency information as training target data based on the preset clean speech database, the clean speech is noise-added, the characteristics of the noise-added speech are extracted, the initial harmonic product spectrum is calculated using short-time Fourier transform and downsampling, and through The improved calculation formula obtains the improved harmonic product spectrum as the model input and trains the fundamental frequency extraction model.
It improves the training efficiency of the fundamental frequency extraction model, reduces the input data dimension, and enhances the real-time and training speed of the model.
Smart Images

Figure CN114067784B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of speech noise reduction, and in particular to a training method and device for a fundamental frequency extraction model, and a fundamental frequency extraction method and device. Background Art
[0002] There is a relatively good cepstrum smoothing technology in speech denoising, which has a good post-processing effect on the pseudo-spectrum and residual between speech harmonics in the neural network model denoising, but it itself relies on a relatively accurate estimation of the speech fundamental frequency (hereinafter referred to as fundamental frequency).
[0003] Because the fundamental frequency extraction methods based on traditional signals, such as those based on correlation, cepstrum, cumulative difference (YIN algorithm), etc., all perform poorly under noise, and even in quiet scenes, due to the harmonic characteristics of speech, it is easy to have half-frequency and double-frequency problems, which often require dynamic programming DP algorithm post-processing to remove wild points, but double-frequency and half-frequency are still two contradictory problems that are difficult to completely overcome at the same time; furthermore, the fundamental frequency extraction methods based on traditional signals often require the use of longer frames to obtain sufficient periodicity, which leads to large prediction delays and is not very suitable for scenarios with high real-time requirements. Related technologies have proposed fundamental frequency extraction methods based on neural network models, which have time domain and frequency domain methods. Although the effect is good, its input dimension is large and modeling is difficult. A large model is required to model it, resulting in a sharp increase in the amount of calculation.
[0004] Therefore, how to improve the training efficiency of the fundamental frequency extraction model is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention
[0005] The present invention provides a training method and device for a fundamental frequency extraction model, and a fundamental frequency extraction method and device, so as to solve the technical problem of low training efficiency of the fundamental frequency extraction model in the prior art.
[0006] The technical solution of the present invention to solve the above technical problems is as follows:
[0007] A training method for a fundamental frequency extraction model, comprising:
[0008] Based on a preset clean speech database, extracting fundamental frequency information of clean speech in the clean speech database as training target data;
[0009] Performing noise processing on the clean speech to obtain noisy speech;
[0010] Extracting features of the noisy speech;
[0011] The features of the noisy speech are used as input data of the model to be trained, and the training target data are used as output data of the model to be trained. The model to be trained is trained to obtain a fundamental frequency extraction model.
[0012] Furthermore, in the training method of the fundamental frequency extraction model described above, extracting the features of the noisy speech includes:
[0013] Calculating the amplitude spectrum characteristics of the noisy speech by framing, windowing, and short-time Fourier transform;
[0014] Downsampling the amplitude spectrum characteristics of the noisy speech to determine an initial harmonic product spectrum of the noisy speech;
[0015] The initial harmonic product spectrum is improved to obtain an improved harmonic product spectrum as a feature of the noisy speech.
[0016] Furthermore, in the training method of the fundamental frequency extraction model described above, downsampling the amplitude spectrum feature of the noisy speech to determine the initial harmonic product spectrum of the noisy speech includes:
[0017] Dividing the noisy speech into frames;
[0018] Obtaining the amplitude spectrum characteristics of the framed speech;
[0019] Downsampling the amplitude spectrum feature of each frame of the framed speech to obtain a downsampled sequence;
[0020] Calculating the initial harmonic product spectrum of the noisy speech according to the downsampled sequence of the amplitude spectrum features of each frame of speech;
[0021] Wherein, the initial harmonic product spectrum is as follows:
[0022] HPS1(K)=S(K)×S(2K)×S(3K)×S(4K)×…S(nK);
[0023] Among them, the HPS1(K) is the initial harmonic product spectrum; the S(K) represents the amplitude spectrum characteristics; and the n represents the order of the harmonic product spectrum.
[0024] Furthermore, in the training method of the fundamental frequency extraction model described above, the initial harmonic product spectrum is improved to obtain an improved harmonic product spectrum as the feature of the noisy speech, including:
[0025] Based on a preset improved harmonic product spectrum calculation formula, the initial harmonic product spectrum is improved to obtain an improved harmonic product spectrum as a feature of the noisy speech;
[0026] The improved harmonic product spectrum calculation formula is:
[0027] HPS2=exp(log(HPS1)-max(log(HPS1)));
[0028] Among them, HPS2 is the improved harmonic product spectrum; log(HPS1) represents the logarithm of the initial harmonic product spectrum, and max(log(HPS1)) represents the maximum value of the logarithm of the initial harmonic product spectrum.
[0029] The present invention also provides a baseband extraction method, comprising:
[0030] Extracting features of speech to be processed;
[0031] Inputting the features of the speech to be processed into a pre-trained fundamental frequency extraction model to extract the fundamental frequency, thereby obtaining the fundamental frequency of the speech to be processed;
[0032] Wherein, the pre-trained fundamental frequency extraction model is obtained by any of the fundamental frequency extraction model training methods described above.
[0033] The present invention also provides a training device for a fundamental frequency extraction model, comprising:
[0034] A first fundamental frequency extraction module, configured to extract fundamental frequency information of clean speech in a preset clean speech database as training target data;
[0035] A noise adding module, used for performing noise adding processing on the clean speech to obtain noisy speech;
[0036] A first feature extraction module, used to extract the features of the noisy speech;
[0037] The training module is used to use the features of the noisy speech as input data of the model to be trained, use the training target data as output data of the model to be trained, train the model to be trained, and obtain a fundamental frequency extraction model.
[0038] Furthermore, in the training device for the fundamental frequency extraction model described above, the first feature extraction module is specifically used to:
[0039] Calculating the amplitude spectrum characteristics of the noisy speech by framing, windowing, and short-time Fourier transform;
[0040] Downsampling the amplitude spectrum characteristics of the noisy speech to determine an initial harmonic product spectrum of the noisy speech;
[0041] The initial harmonic product spectrum is improved to obtain an improved harmonic product spectrum as a feature of the noisy speech.
[0042] Furthermore, in the training device for the fundamental frequency extraction model described above, the second extraction module is further used to divide the noisy speech into frames;
[0043] Obtaining the amplitude spectrum characteristics of the framed speech;
[0044] Downsampling the amplitude spectrum feature of each frame of the framed speech to obtain a downsampled sequence;
[0045] Calculating the initial harmonic product spectrum of the noisy speech according to the downsampled sequence of the amplitude spectrum features of each frame of speech;
[0046] Wherein, the initial harmonic product spectrum is as follows:
[0047] HPS1(K)=S(K)×S(2K)×S(3K)×S(4K)×…S(nK);
[0048] Among them, the HPS1(K) is the initial harmonic product spectrum; the S(K) represents the amplitude spectrum characteristics; and the n represents the order of the harmonic product spectrum.
[0049] Furthermore, in the training device for the fundamental frequency extraction model described above, the second extraction module is further used for:
[0050] Based on a preset improved harmonic product spectrum calculation formula, the initial harmonic product spectrum is improved to obtain an improved harmonic product spectrum as a feature of the noisy speech;
[0051] The improved harmonic product spectrum calculation formula is:
[0052] HPS2=exp(log(HPS1)-max(log(HPS1)));
[0053] Among them, HPS2 is the improved harmonic product spectrum; log(HPS1) represents the logarithm of the initial harmonic product spectrum, and max(log(HPS1)) represents the maximum value of the logarithm of the initial harmonic product spectrum.
[0054] The present invention also provides a baseband extraction device, comprising:
[0055] A second feature extraction module is used to extract features of the speech to be processed;
[0056] A second fundamental frequency extraction module is used to input the features of the speech to be processed into a pre-trained fundamental frequency extraction model to extract the fundamental frequency, so as to obtain the fundamental frequency of the speech to be processed;
[0057] Wherein, the pre-trained fundamental frequency extraction model is obtained by any of the fundamental frequency extraction model training methods described above.
[0058] The beneficial effects of the present invention are:
[0059] By extracting the fundamental frequency information of clean speech from a preset clean speech database as training target data, and using the training target data as the output data of the model to be trained, after adding noise to the clean speech to obtain the noisy speech data, the features of the noisy speech are extracted as the input data of the model to be trained, the dimensionality reduction processing of the input data of the training model is achieved, and the known output data is used for training, so that the training efficiency of the fundamental frequency extraction model is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1 A flowchart of an embodiment of a training method for a fundamental frequency extraction model of the present invention;
[0061] Figure 2 is a flow chart of an embodiment of a baseband extraction method of the present invention;
[0062] Figure 3 It is a structural schematic diagram of an embodiment of a training device for a fundamental frequency extraction model of the present invention;
[0063] Figure 4 It is a structural schematic diagram of an embodiment of a training device for a fundamental frequency extraction model of the present invention;
[0064] Figure 5 It is a schematic structural diagram of the electronic device of the present invention. DETAILED DESCRIPTION
[0065] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.
[0066] Figure 1 FIG. 1 is a flow chart of an embodiment of a training method for a fundamental frequency extraction model of the present invention, as shown in FIG. Figure 1 As shown, the training method of the fundamental frequency extraction model of this embodiment may specifically include the following steps:
[0067] 100. Based on a preset clean speech database, extracting fundamental frequency information of clean speech in the clean speech database as training target data;
[0068] In a specific implementation process, the signal-based fundamental frequency detection tool usually has a higher accuracy in a quiet scene. Therefore, in this embodiment, the fundamental frequency extraction tool v_fxpefac of the voice toolbox in the related technology (the voicebox toolbox known in the field) can be used to extract the fundamental frequency information of the clean speech from the clean speech in the clean speech database for use as training target data.
[0069] In practical applications, considering that the fundamental frequency of human speech is mostly concentrated in 80-400 Hz, usually 60-dimensional frequency domain coordinates can cover the fundamental frequency of human speech. Therefore, through this tool, each speech corresponds to multiple frames, and each frame has a fundamental frequency, which can be mapped to the 60-dimensional frequency domain coordinates by rounding the fundamental frequency.
[0070] 101. Performing noise processing on the clean speech to obtain noisy speech;
[0071] In a specific implementation process, the clean speech may be subjected to noise adding processing by adding noise or the like to obtain the noisy speech.
[0072] 102. Extracting features of the noisy speech;
[0073] In a specific implementation process, the features of the noisy speech can be extracted by the following steps:
[0074] (1) calculating the amplitude spectrum characteristics of the noisy speech by framing, windowing, and short-time Fourier transform;
[0075] (2) downsampling the amplitude spectrum characteristics of the noisy speech to determine the initial harmonic product spectrum of the noisy speech;
[0076] Specifically, the noisy speech is framed through the frequency band representation; the amplitude spectrum characteristics of the speech after framing are obtained; the amplitude spectrum characteristics of each frame of the speech after framing are downsampled to obtain a downsampled sequence; and the initial harmonic product spectrum of the noisy speech is calculated based on the downsampled sequence of the amplitude spectrum characteristics of each frame of speech.
[0077] Wherein, the initial harmonic product spectrum is as follows:
[0078] HPS1(K)=S(K)×S(2K)×S(3K)×S(4K)×…S(nK);
[0079] Wherein, the HPS1(K) is the initial harmonic product spectrum; the S(K) represents the amplitude spectrum feature; and the n represents the harmonic product spectrum order. In this embodiment, n may be equal to 4.
[0080] (3) Improving the initial harmonic product spectrum to obtain an improved harmonic product spectrum as a feature of the noisy speech.
[0081] In a specific implementation process, the initial harmonic product spectrum may be improved based on a preset improved harmonic product spectrum calculation formula to obtain an improved harmonic product spectrum as the feature of the noisy speech.
[0082] Wherein, the improved harmonic product spectrum calculation formula is:
[0083] HPS2=exp(log(HPS1)-max(log(HPS1)));
[0084] Among them, HPS2 is the improved harmonic product spectrum; log(HPS1) represents the logarithm of the initial harmonic product spectrum, and max(log(HPS1)) represents the maximum value of the logarithm of the initial harmonic product spectrum.
[0085] In a specific implementation process, since speech energy varies, the improved harmonic product spectrum obtained by improving the harmonic product spectrum calculation formula has nothing to do with the absolute energy, but the harmonic characteristics are still maintained and it is also easy to train.
[0086] 103. Using the features of the noisy speech as input data of the model to be trained, using the training target data as output data of the model to be trained, training the model to be trained, and obtaining a fundamental frequency extraction model.
[0087] In a specific implementation process, the RNN-Noise model of the related technology can be improved, the characteristics of the noisy speech are used as the input data of the model to be trained, the training target data is used as the output data of the model to be trained, the model to be trained is trained, and the fundamental frequency extraction model is obtained.
[0088] It should be noted that the voice activity detection output content corresponding to the RNN-Noise model can also be modified to voiced consonants to further ensure that the fundamental frequency of the speech signal can be extracted while filtering the fundamental frequency of the noise signal.
[0089] In a specific implementation process, the time domain and frequency domain inputs of the fundamental frequency extraction method based on the neural network model in the related art are both a complete frame of signal, and through the solution of HPS2, the fundamental frequency information will be highlighted, which is easier to learn, and only the features of the fundamental frequency distribution interval can be selected, so the amount of input data is much smaller than the time domain and frequency domain inputs of the fundamental frequency extraction method based on the neural network model in the related art. In other words, the present application uses lower-dimensional input data and known output data for training, which greatly improves the training efficiency of the fundamental frequency extraction model.
[0090] The training method of the fundamental frequency extraction model of the present embodiment extracts the fundamental frequency information of clean speech from a preset clean speech database as training target data, and uses the training target data as output data of the model to be trained. After adding noise to the clean speech to obtain the noisy speech data, the features of the noisy speech are extracted as input data of the model to be trained, thereby achieving dimensionality reduction processing of the input data of the model to be trained, and using known output data for training, so that the training efficiency of the fundamental frequency extraction model is greatly improved.
[0091] It should be noted that the method of the embodiment of the present invention can be performed by a single device, such as a computer or a server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present invention, and the multiple devices will interact with each other to complete the method.
[0092] Figure 2 is a flow chart of an embodiment of a baseband extraction method of the present invention, as shown in Figure 2 As shown, the baseband extraction method of this embodiment may specifically include the following steps:
[0093] 200. Extracting features of the speech to be processed;
[0094] 201. Input the features of the speech to be processed into a pre-trained fundamental frequency extraction model to extract the fundamental frequency, so as to obtain the fundamental frequency of the speech to be processed.
[0095] In a specific implementation process, the pre-trained fundamental frequency extraction model is obtained by the fundamental frequency extraction model training method of the above embodiment.
[0096] Figure 3 FIG. 1 is a structural diagram of an embodiment of a training device for a fundamental frequency extraction model of the present invention, as shown in FIG. Figure 3 As shown, the training device for the fundamental frequency extraction model of this embodiment may include a first fundamental frequency extraction module 30 , a noise adding module 31 , a first feature extraction module 32 and a training module 33 .
[0097] A first fundamental frequency extraction module 30 is used to extract fundamental frequency information of clean speech in a preset clean speech database as training target data;
[0098] A noise adding module 31, used for performing noise adding processing on the clean speech to obtain a noisy speech;
[0099] A first feature extraction module 32, used to extract the features of the noisy speech;
[0100] Specifically, the first feature extraction module 32 can extract the features of the noisy speech through the following steps:
[0101] (1) calculating the amplitude spectrum characteristics of the noisy speech by framing, windowing, and short-time Fourier transform;
[0102] (2) downsampling the amplitude spectrum characteristics of the noisy speech to determine the initial harmonic product spectrum of the noisy speech;
[0103] Specifically, the noisy speech is framed through the frequency band representation; the amplitude spectrum characteristics of the speech after framing are obtained; the amplitude spectrum characteristics of each frame of the speech after framing are downsampled to obtain a downsampled sequence; and the initial harmonic product spectrum of the noisy speech is calculated based on the downsampled sequence of the amplitude spectrum characteristics of each frame of speech.
[0104] Wherein, the initial harmonic product spectrum is as follows:
[0105] HPS1(K)=S(K)×S(2K)×S(3K)×S(4K)×…S(nK);
[0106] Among them, the HPS1(K) is the initial harmonic product spectrum; the S(K) represents the amplitude spectrum characteristics; and the n represents the order of the harmonic product spectrum.
[0107] (3) Improving the initial harmonic product spectrum to obtain an improved harmonic product spectrum as a feature of the noisy speech.
[0108] Specifically, the initial harmonic product spectrum may be improved based on a preset improved harmonic product spectrum calculation formula to obtain an improved harmonic product spectrum as the feature of the noisy speech;
[0109] Wherein, the improved harmonic product spectrum calculation formula is:
[0110] HPS2=exp(log(HPS1)-max(log(HPS1)));
[0111] Among them, HPS2 is the improved harmonic product spectrum; log(HPS1) represents the logarithm of the initial harmonic product spectrum, and max(log(HPS1)) represents the maximum value of the logarithm of the initial harmonic product spectrum.
[0112] The training module 33 is used to use the features of the noisy speech as input data of the model to be trained, and the training target data as output data of the model to be trained, to train the model to be trained and obtain a fundamental frequency extraction model.
[0113] Figure 4 FIG. 1 is a structural diagram of an embodiment of a training device for a fundamental frequency extraction model of the present invention, as shown in FIG. Figure 4 As shown, the training device for the fundamental frequency extraction model of this embodiment may include a second feature extraction module 40 and a second fundamental frequency extraction module 41.
[0114] A second feature extraction module 40, used to extract features of the speech to be processed;
[0115] A second fundamental frequency extraction module 41 is used to input the features of the speech to be processed into a pre-trained fundamental frequency extraction model to extract the fundamental frequency, so as to obtain the fundamental frequency of the speech to be processed;
[0116] The pre-trained fundamental frequency extraction model is obtained by the fundamental frequency extraction model training method of the above embodiment.
[0117] The device of the above embodiment is used to implement the corresponding method in the above embodiment. Its specific implementation scheme can refer to the method recorded in the above embodiment and the relevant description in the method embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.
[0118] Figure 5 is a schematic diagram of the structure of the electronic device of the present invention, such as Figure 5 As shown, the electronic device of this embodiment may include: a processor 1010 and a memory 1020. Those skilled in the art will appreciate that the device may also include an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other in communication within the device via the bus 1050.
[0119] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions of the training method of the baseband extraction model and the baseband extraction method provided in the embodiments of this specification.
[0120] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.
[0121] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0122] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).
[0123] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).
[0124] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.
[0125] The present invention also provides a storage medium storing one or more programs, which can be executed to implement the training method of the fundamental frequency extraction model and the fundamental frequency extraction method of the above-mentioned embodiment.
[0126] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0127] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Within the scope of the present invention, the above embodiments or technical features in different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity.
[0128] In addition, to simplify the description and discussion, and in order not to obscure the present invention, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the devices may be shown in the form of block diagrams to avoid obscuring the present invention, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the present invention will be implemented (i.e., these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present invention, it will be apparent to those skilled in the art that the present invention may be implemented without these specific details or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0129] Although the invention has been described in conjunction with specific embodiments of the invention, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
[0130] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A training method for a fundamental frequency extraction model, characterized in that: include: Based on a preset clean speech database, extracting fundamental frequency information of clean speech in the clean speech database as training target data; Performing noise processing on the clean speech to obtain noisy speech; Calculating the amplitude spectrum characteristics of the noisy speech by framing, windowing, and short-time Fourier transform; Downsampling the amplitude spectrum characteristics of the noisy speech to determine an initial harmonic product spectrum of the noisy speech; Based on a preset improved harmonic product spectrum calculation formula, the initial harmonic product spectrum is improved to obtain an improved harmonic product spectrum as the feature of the noisy speech; wherein the improved harmonic product spectrum calculation formula is: HPS2=exp(log(HPS1)-max(log(HPS1))); wherein HPS2 is the improved harmonic product spectrum; log(HPS1) represents the logarithm of the initial harmonic product spectrum, and max(log(HPS1)) represents the maximum value of the logarithm of the initial harmonic product spectrum; The features of the noisy speech are used as input data of the model to be trained, and the training target data are used as output data of the model to be trained. The model to be trained is trained to obtain a fundamental frequency extraction model.
2. The training method of the fundamental frequency extraction model according to claim 1, characterized in that: Downsampling the amplitude spectrum feature of the noisy speech to determine the initial harmonic product spectrum of the noisy speech includes: Dividing the noisy speech into frames; Obtaining the amplitude spectrum characteristics of the framed speech; Downsampling the amplitude spectrum feature of each frame of the framed speech to obtain a downsampled sequence; Calculating the initial harmonic product spectrum of the noisy speech according to the downsampled sequence of the amplitude spectrum features of each frame of speech; Wherein, the initial harmonic product spectrum is as follows: HPS1(K)=S(K)×S(2K)×S(3K)×S(4K)×…S(nK); Among them, the HPS1(K) is the initial harmonic product spectrum; the S(K) represents the amplitude spectrum characteristics; and the n represents the order of the harmonic product spectrum.
3. A baseband extraction method, characterized in that: include: Extracting features of speech to be processed; Inputting the features of the speech to be processed into a pre-trained fundamental frequency extraction model to extract the fundamental frequency, thereby obtaining the fundamental frequency of the speech to be processed; Wherein, the pre-trained fundamental frequency extraction model is obtained by the fundamental frequency extraction model training method described in claim 1 or 2.
4. A training device for a fundamental frequency extraction model, characterized in that: include: A first fundamental frequency extraction module, configured to extract fundamental frequency information of clean speech in a preset clean speech database as training target data; A noise adding module, used for performing noise adding processing on the clean speech to obtain noisy speech; A first feature extraction module, used for calculating the amplitude spectrum features of the noisy speech through framing, windowing, and short-time Fourier transform; Down-sampling the amplitude spectrum feature of the noisy speech to determine the initial harmonic product spectrum of the noisy speech; based on a preset improved harmonic product spectrum calculation formula, improving the initial harmonic product spectrum to obtain an improved harmonic product spectrum as the feature of the noisy speech; wherein the improved harmonic product spectrum calculation formula is: HPS2 = exp(log(HPS1)-max(log(HPS1))); wherein HPS2 is the improved harmonic product spectrum; log(HPS1) represents the logarithm of the initial harmonic product spectrum, and max(log(HPS1)) represents the maximum value of the logarithm of the initial harmonic product spectrum; The training module is used to use the features of the noisy speech as input data of the model to be trained, use the training target data as output data of the model to be trained, train the model to be trained, and obtain a fundamental frequency extraction model.
5. The training device for the fundamental frequency extraction model according to claim 4, characterized in that: The first feature extraction module is further used to divide the noisy speech into frames; Obtaining the amplitude spectrum characteristics of the framed speech; Downsampling the amplitude spectrum feature of each frame of the framed speech to obtain a downsampled sequence; Calculating the initial harmonic product spectrum of the noisy speech according to the downsampled sequence of the amplitude spectrum features of each frame of speech; Wherein, the initial harmonic product spectrum is as follows: HPS1(K)=S(K)×S(2K)×S(3K)×S(4K)×…S(nK); Among them, the HPS1(K) is the initial harmonic product spectrum; the S(K) represents the amplitude spectrum characteristics; and the n represents the order of the harmonic product spectrum.
6. A fundamental frequency extraction device, characterized in that: include: A second feature extraction module is used to extract features of the speech to be processed; A second fundamental frequency extraction module is used to input the features of the speech to be processed into a pre-trained fundamental frequency extraction model to extract the fundamental frequency, so as to obtain the fundamental frequency of the speech to be processed; Wherein, the pre-trained fundamental frequency extraction model is obtained by the fundamental frequency extraction model training method described in claim 1 or 2.
