Speech signal analysis and synthesis method, electronic device, and storage medium

Abstract

The application discloses a speech signal analysis and synthesis method, an electronic device and a storage medium, wherein the method comprises the following steps: obtaining a mel spectrum, converting the mel spectrum into a source signal through a sinusoidal model; performing frame division on the source signal, and inputting the frame-divided source signal and the mel spectrum into a neural network model to obtain an output result of the neural network model, wherein the frame-divided source signal and the mel spectrum are of the same frame level; and obtaining a final speech through at least a first-order filter bank. According to the application, the mel spectrum is converted into the source signal through the sinusoidal model, and the source signal is combined with the mel spectrum to be input into the neural network model for processing, so that the metallic feeling of the synthesized sound at the clear and dull sound boundary can be removed, the calculation amount of the neural network is reduced, and the calculation amount is reduced through the reduction of the frequency of the neural network output.

Description

Technical Field

[0001] This invention belongs to the field of speech processing technology, and particularly relates to a speech signal analysis and synthesis method, electronic device, and storage medium. Background Technology

[0002] In existing technologies, similar technologies mainly include Neural Source-Filter (NSF) vocoders. NSF vocoders are neural network vocoders based on a source-filter model. They can be divided into two parts: a sine wave generator and a deep convolutional neural network. The input of NSF is the frame-level fundamental frequency and mel spectrum. The sine wave generator module generates a time-domain sine wave signal after inputting the fundamental frequency. The sine wave signal and the mel spectrum are converted into the final time-domain speech signal through the neural network, achieving the purpose of nonlinear filtering. NSF has the following characteristics: (1) It is a non-autoregressive vocoder, so it is fast to compute on a GPU; (2) It is a generative model trained based on GAN, which has the characteristic of high synthesized sound quality; (3) Based on the source-filter model, the phase continuity of the generated signal is guaranteed by the source signal.

[0003] Existing NSF vocoders are computationally intensive because the input and output of the neural network in NSF are both time-domain signals. For example, generating one second of 24kHz audio using hn-NSF requires approximately 57.6 x 10⁹ floating-point operations. In vocoder research, high-quality vocoders generally require significant computation, increasing the cost of deploying speech synthesis services. The reason for the defects in NSF's sound quality is that the NSF source signal is generated based on a simple voice-unvoice model, which assumes that the speech signal has only two states: unvoiced segments and voiced segments. However, real speech signals often have a quasi-periodic signal at the boundary between unvoiced and voiced segments. This signal cannot be well modeled by this model, so the synthesized sound at the boundary between unvoiced and voiced segments often has a metallic quality.

[0004] The inventors discovered that the input and output of existing (NSF) vocoders are time-domain audio, which requires a lot of computation when synthesizing high-quality speech (such as 24kHz audio). The convolutional neural network used by NSF has a very high number of layers. At the same time, the audio output of (NSF) vocoders often has a semi-periodic signal at the junction of voiced and unvoiced sounds, which makes the synthesized sound at the boundary of voiced and unvoiced sounds have a metallic texture. Summary of the Invention

[0005] The embodiments of the present invention are intended to solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, embodiments of the present invention provide a speech signal analysis and synthesis method, comprising: acquiring a Mel spectrum; converting the Mel spectrum into a source signal using a sine model; framing the source signal; inputting the framed source signal and the Mel spectrum into a neural network model to obtain the output of the neural network model, wherein the framed source signal and the Mel spectrum are at the same frame level; and passing the output through at least a first-order filter bank to obtain the final speech.

[0007] In a second aspect, embodiments of the present invention provide an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described speech signal analysis and synthesis methods of the present invention.

[0008] Thirdly, embodiments of the present invention provide a storage medium storing one or more programs including execution instructions, which can be read and executed by electronic devices (including but not limited to computers, servers, or network devices) to perform any of the above-described speech signal analysis and synthesis methods of the present invention.

[0009] Fourthly, embodiments of the present invention also provide a computer program product, the computer program product including a computer program stored on a storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to execute any of the above-described speech signal analysis and synthesis methods.

[0010] This invention utilizes a sinusoidal model to convert the Mel spectrum into a source signal, and then inputs the source signal and the Mel spectrum into a neural network model for processing. This removes the metallic quality of synthesized sounds at the boundary between voiced and unvoiced tones, while reducing the computational load of the neural network. The computational load is reduced by lowering the frequency of the neural network output. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of an embodiment of the speech signal analysis and synthesis method of the present invention;

[0013] Figure 2 This is a flowchart of another embodiment of the speech signal analysis and synthesis method of the present invention;

[0014] Figure 3 This is a schematic diagram of the sinusoidal model structure of the speech signal analysis and synthesis method of the present invention;

[0015] Figure 4 This is a schematic diagram of the neural network structure of the speech signal analysis and synthesis method of the present invention;

[0016] Figure 5 This is a schematic diagram of the vocoder comparison effect of the speech signal analysis and synthesis method of the present invention.

[0017] Figure 6 A flowchart of a speech signal analysis and synthesis process is provided in one embodiment of the present invention;

[0018] Figure 7 This is a schematic diagram of the structure of an embodiment of the electronic device of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0021] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, elements, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0022] In this invention, terms such as "module," "device," and "system" refer to relevant entities applied to a computer, such as hardware, combinations of hardware and software, software, or software in execution. More specifically, for example, an element can be, but is not limited to, a process running on a processor, a processor, an object, an executable element, an execution thread, a program, and / or a computer. Furthermore, an application program or script running on a server, and the server itself, can also be an element. One or more elements may be in an execution process and / or thread, and elements may be localized on a single computer and / or distributed across two or more computers, and may be run on various computer-readable media. Elements can also communicate via local and / or remote processes based on signals having one or more data packets, for example, signals from data interacting with another element in a local system, a distributed system, and / or interacting with other systems via signals over a network of the Internet.

[0023] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising" or "including" include not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0024] This invention provides a method for analyzing and synthesizing speech signals, which can be applied to electronic devices. The electronic device can be a computer, server, or other electronic product, etc., and this invention does not limit this to any particular device.

[0025] Please refer to Figure 1 This illustrates a speech signal analysis and synthesis method provided by an embodiment of the present invention.

[0026] like Figure 1 As shown, in step 101, the Mel spectrum is obtained, and the Mel spectrum is converted into a source signal using a sine model;

[0027] In step 102, the source signal is framed, and the framed source signal and the Mel spectrum are input into the neural network model to obtain the output result of the neural network model. The framed source signal and the Mel spectrum are at the same frame level.

[0028] In step 103, the output result is passed through at least a first-order filter bank to obtain the final speech.

[0029] In this embodiment, for step 101, the Mel spectrum in the speech signal is obtained, and then the Mel spectrum in the speech signal is converted into the source signal after a series of processing using a sine model. Using a sine model to convert the Mel spectrum into the source signal can effectively remove the metallic texture of the synthesized sound at the boundary between voiced and unvoiced sounds. The Mel scale is a roughly equidistant division of the frequency scale according to the human ear's perception of "pitch," and its basis comes from a historical experiment on human hearing. For a time-domain signal, its Mel spectrum is the result of its short-time Fourier transform distorted to the Mel scale. Its calculation method can be simply described as calculating the short-time Fourier transform of the time-domain signal, and then multiplying the result of the short-time Fourier transform of each frame by the transform matrix M representing the Mel filter bank (in actual calculation, this matrix can use the open-source library librosa.filters.mel) to obtain the Mel spectrum.

[0030] Next, in step 102, the converted source signal undergoes frame segmentation, also known as a folding operation. The source signal is folded into frame levels identical to those in the Mel spectrum. The folded source signal and the Mel spectrum are then simultaneously input into a neural network model for processing. The output of the neural network model after processing the folded source signal and the Mel spectrum is obtained. Since the folded source signal and the Mel spectrum are at the same frame level, the source signal is folded and divided into a sequence of frame levels identical to those in the Mel spectrum.

[0031] Finally, for step 103, the output of the neural network is input into a PQMF (Pseudo-Quadrature Mirror Filter) filter bank with minimal signal reconstruction error for processing. In this application, a 4th-order PQMF filter bank is used. The output of the neural network is passed through the 4th-order PQMF filter bank to obtain the final speech.

[0032] The method in this application embodiment uses a sine model to convert the Mel spectrum into a source signal, and then inputs the source signal and Mel spectrum into a neural network model for processing. This can remove the metallic quality of the synthesized sound at the boundary between voiced and unvoiced tones, while reducing the computational load of the neural network. By reducing the frequency of the neural network output, the computational load is reduced.

[0033] Please refer to Figure 2 This illustrates another speech signal analysis and synthesis method provided by an embodiment of the present invention. The flowchart is primarily a flowchart... Figure 1 The flowchart further defines the step of "converting the Mel spectrum into the source signal through a sine model" in step 101.

[0034] like Figure 2As shown, in step 201, the Mel spectrum is converted into at least one frame of linear spectrum using an inverse Mel filter and the spectral peaks in the at least one frame of linear spectrum are obtained.

[0035] In step 202, all the spectral peaks are matched to obtain peak orbital groups;

[0036] In step 203, at least one group of the wave crest orbits is sampled, and the phase of the sampled wave crest orbits is calculated to obtain the source signal.

[0037] In this embodiment, for step 201, the Mel spectrum is converted into a linear spectrum using an inverse Mel filter. Based on the converted linear spectrum, the maximum value of the linear spectrum energy in at least one frame is obtained, i.e., the spectral peak. A maximum value search is performed to find the maximum value of the linear spectrum energy in each frame. To convert the Mel spectrum into a linear spectrum, a matrix L is found such that the calculated LM matrix approximates the identity matrix E. In the actual algorithm, the approximate inverse transformation algorithm of numpy.linalg.pinv is used, taking L = pinv(M), and then the matrix X ∈ R representing the Mel spectrum is... N ×T Multiplying by the L matrix yields the linear spectrum Y = LX.

[0038] Next, in step 202, after obtaining the spectral peaks in the linear spectrum, all spectral peaks are matched. The linear spectra of all frames are sorted according to their maxima (spectral peaks), except for the last frame. Based on the sorted linear spectrum of each frame, a simple greedy algorithm is used to find the spectral peaks with the closest frequencies for matching, obtaining the matching results. Only matching results with frequency differences and peak energy differences less than a certain threshold are retained. The matching results are the spectral peak matching results of adjacent frames. In fact, a group of peak tracks with continuous time, amplitude, and frequency is found on the complete time axis, which is equivalent to obtaining the peak track group after obtaining the matching results. Tracks with too short a duration are deleted according to a certain threshold based on the duration (if a peak does not have a corresponding match in the previous frame, it is the start of a track; if it does not have a corresponding match in the next frame, it is the end of a track).

[0039] Finally, for step 203, at least one set of peak tracks is sampled. For each continuous peak track in terms of time, amplitude, and frequency within the peak track set, the energy and frequency of the peak are sampled using linear interpolation to a sampling rate consistent with the synthesized speech (e.g., 24kHz). After sampling, the phase of at least one set of peak tracks is calculated. For example, for the sampled peak tracks, since the instantaneous frequencies of the tracks at different times are already known, the phase of the track is calculated at each moment within the duration with an initial phase of 0, and the end time of the track is recalibrated by returning to 0 with the last phase within the duration. After obtaining the phase, a total output signal is obtained by calculating using a sine function; this output signal is the source signal.

[0040] The method in this application embodiment obtains the source signal for speech signal analysis and synthesis using a sine model. This method can effectively remove the metallic texture of synthesized sounds at the boundary between voiced and unvoiced sounds.

[0041] In some optional embodiments, the spectral peaks of each frame are sorted, and spectral peaks with similar frequencies in each frame are matched. For example, for each frame (except the last frame), the spectral peaks are sorted, and a simple greedy algorithm is used to find the peak with the closest frequency for each current frame's spectral peaks for matching. In the final matching results, only matches where the frequency difference and peak energy difference are less than a certain threshold are retained. If the frequency difference and peak energy difference are less than a certain threshold, they are used. Only matches with values ​​less than the threshold are retained. This matching result is a group of peak tracks that are continuous in time, amplitude, and frequency. According to the sinusoidal model's assumption about speech signals, speech signals consist of a set of sinusoidal signals with different frequencies and amplitudes that change continuously in time. When actually calculating the parameters of the sinusoidal model, it is necessary to further add the constraint that "each sinusoidal signal changes slowly" to make the calculated parameters sufficiently stable and avoid introducing a large amount of noise when reconstructing the time-domain signal.

[0042] In some optional embodiments, the energy and frequency of the spectral peaks in each peak track group are sampled using linear interpolation until the sampling rate matches that of the synthesized speech, where the synthesized speech has a sampling rate of 24 kHz. Assuming the sinusoidal model uses a set of parameters every 10 milliseconds (i.e., the frequencies and amplitudes of N sine waves), and the desired final synthesization is 16 kHz speech (i.e., one sampling point every 1 / 16 milliseconds), then we need to upsample the parameters of the sinusoidal model by a factor of 160. We simply insert 159 new sets of parameters Y between two adjacent sets of sinusoidal parameters X0 and X1. t (t=1.159), and let Y t = (t·X1+(160-t)·X0) / 160, which means upsampling is achieved by linear interpolation.

[0043] In some optional embodiments, the phase of the crest track at each moment within the duration is calculated based on a preset initial phase, and the end time of the track is recalibrated by returning to the preset initial phase with the last phase within the duration, where the preset initial phase is 0. For example, for a sampled crest track, since the instantaneous frequencies of the track at different moments are already known, we calculate the phase of the track at each moment within the duration with an initial phase of 0, and recalibrate the end time of the track by returning to 0 with the last phase within the duration. After recalibrating the end time of the track, the amplitude value of the crest track at each moment is calculated using a sine function, and all crest tracks are superimposed to obtain an output signal, which is the source signal. For example, after obtaining the phase p(t) and each continuous crest track in terms of time, amplitude, and frequency, the amplitude value of the track at each moment is calculated using the sine function y(t) = A(t)·s in(p(t)), and all tracks are superimposed to obtain a total output signal Y(t) = ∑y(t), where the total output signal Y(t) = ∑y(t) is the source signal.

[0044] Furthermore, the source signal is processed by low-pass filtering, which divides the source signal into a frame-level sequence that is the same as the Mel spectrum feature frame.

[0045] It should be noted that the solution in this application preserves the source signal and advantages of NSF while reducing the computational cost of the vocoder and addressing the issue of metallic quality in synthesized sounds at the voiced / unvoiced boundary. First, a sinusoidal model is chosen to replace the original simple binary source model. The sinusoidal model does not rely on the assumption that "speech only has two states: voiced and unvoiced," but instead treats the speech signal as a superposition of N sinusoidal signals, thus resolving the sound quality issue at the voiced / unvoiced boundary. Second, a 2MelGAN neural network structure is used. Finally, a PQMF filter bank is used to filter the output of the neural network, thereby reducing the frequency of the neural network output and further reducing the computational cost.

[0046] It should be noted that this application also provides an alternative solution: replacing the neural network model with HiFiGAN or other GAN-based non-autoregressive vocoder neural network structures. The neural network used in this application is characterized by low computational cost and high sound quality fidelity. Replacing it with other neural network structures sacrifices computational cost for higher fidelity, or reduces computational cost at the expense of sound quality. The sine model in this application can be replaced with a harmonic plus noise model in specific scenarios. Its advantage is that it may perform better on certain specific data (e.g., f0 extraction of speakers with very accurate speech), but its versatility is poor. Using multiple neural networks, with the output of the previous neural network folded and used as the source signal input of the next neural network, while the Mel spectrum serves as the common input of all used neural networks. The advantage is that the computational cost of the model can be flexibly adjusted, allowing the use of an appropriate number of neural networks on the corresponding machine to achieve the best synthesis effect. The disadvantage of this approach is that the training steps for the neural networks are numerous, increasing the development cycle.

[0047] Please refer to Figure 3 The diagram shows a sinusoidal model structure of the speech signal analysis and synthesis method of the present invention.

[0048] like Figure 3 As shown, step 1: The Mel spectrum is converted into a linear spectrum using an inverse Mel filter;

[0049] Step 2: Maximal search to find the maximum value of spectral energy (i.e., spectral peak) in each frame;

[0050] Step 3: For each frame (except the last frame), sort by maximum value, and use a simple greedy algorithm to find the peak with the closest frequency for each current frame's spectrum peaks for matching. In the final matching results, we only retain the matches where the frequency difference and peak energy difference between the two matches are less than a certain threshold.

[0051] Step 4: After finding the peak matching of adjacent frames in Step 3, we have actually found a group of peak tracks with continuous time, amplitude and frequency on the complete time axis. We delete tracks with too short a duration according to a certain threshold based on the duration (if a peak does not have a corresponding match in the previous frame, it is the beginning of a track; if it does not have a corresponding match in the next frame, it is the end of a track).

[0052] Step 5: For each continuous peak track in terms of time, amplitude, and frequency, upsample the energy and frequency of the peak to a sampling rate consistent with the synthesized speech (e.g., 24kHz) using linear interpolation.

[0053] Step 6: For the sampled peak track, since we already have the instantaneous frequency of the track at different times, we calculate the phase of the track at each moment within the duration using an initial phase of 0, and then recalibrate the end time of the track by returning to 0 with the last phase within the duration. Before and after obtaining the phase p(t) and the amplitude A(t) from Step 5, we calculate the amplitude value of the track at each moment using the sine function y(t) = A(t)·sin(p(t)), and then superimpose all the tracks to obtain a total output signal Y(t) = ∑y(t).

[0054] Please refer to Figure 4 The paper presents a schematic diagram of the neural network structure of the speech signal analysis and synthesis method of the present invention.

[0055] like Figure 4 As shown, the folded source signal and the Mel spectrum are simultaneously input into the neural network.

[0056] Please refer to Figure 5 The diagram shows the vocoder comparison effect of the speech signal analysis and synthesis method of the present invention.

[0057] like Figure 5 As shown, (vocoder step 1) the sinusoidal model converts the Mel spectrum into the source signal.

[0058] Effect 1: Removes the metallic quality of synthesized sounds at the voicing / unvoicing boundary. Because a sinusoidal model is used instead of a simple voiced-unvoiced binary model, it provides better modeling of the half-cycle signal at the voicing / unvoicing boundary.

[0059] (Vocoder step 2) Perform low-pass filtering on the source signal.

[0060] Effect 2: The synthesized sound is more stable because the low-pass filter eliminates high-frequency signals / noise.

[0061] (Vocoder step 3) Fold the source signal.

[0062] Effect 3: Reduces the computational cost of neural networks because the input sampling rate is reduced to the frame level.

[0063] (Vocoder step 4) Use a neural network as a nonlinear filter.

[0064] Effect 4: It retains the advantages of high fidelity of neural network vocoders and the temporal continuity of the generated signal guaranteed by the vocal model.

[0065] (Vocoder step 5) Use a 4th order PQMF filter.

[0066] Effect 5: Further reduce the computational cost of the neural network. By reducing the frequency of the neural network output, the computational cost is reduced.

[0067] Please refer to Figure 6 The document presents a flowchart illustrating the implementation of the speech signal analysis and synthesis method of the present invention.

[0068] like Figure 6 As shown, step 1: convert the Mel spectrum into the source signal using a sinusoidal model;

[0069] Step 2: Perform low-pass filtering on the source signal;

[0070] Step 3: "Fold" the source signal, i.e., perform a framing operation, dividing it into a frame-level sequence with the same frame shift as the Mel spectrum;

[0071] Step 4: Input the folded source signal and Mel spectrum simultaneously into the neural network. This step uses a Style MelGAN neural network. The source signal input replaces the noise input in the original Style MelGAN paper.

[0072] Step 5: Pass the output of the neural network through a 4th-order PQMF filter bank to obtain the final speech.

[0073] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of combined actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, as some steps can be performed in other orders or simultaneously according to the present invention. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention. In the above embodiments, the descriptions of each embodiment have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0074] In some embodiments, the present invention provides a non-volatile computer-readable storage medium storing one or more programs including execution instructions, which can be read and executed by electronic devices (including but not limited to computers, servers, or network devices) to perform any of the above-described speech signal analysis and synthesis methods of the present invention.

[0075] In some embodiments, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-volatile computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform any of the above-described speech signal analysis and synthesis methods.

[0076] In some embodiments, the present invention also provides an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a speech signal analysis and synthesis method.

[0077] Figure 7 This is a schematic diagram of the hardware structure of an electronic device for performing a speech signal analysis and synthesis method according to another embodiment of this application, as shown below. Figure 7 As shown, the device includes:

[0078] One or more processors 710 and memory 720, Figure 7 Take the 710 processor as an example.

[0079] The device for performing the speech signal analysis and synthesis method may further include an input device 730 and an output device 740.

[0080] The processor 710, memory 720, input device 730, and output device 740 can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0081] The memory 720, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the speech signal analysis and synthesis method in the embodiments of this application. The processor 710 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 720, thereby implementing the speech signal analysis and synthesis method of the above-described method embodiments.

[0082] The memory 720 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the speech signal analysis and synthesis device. Furthermore, the memory 720 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 720 may optionally include memory remotely located relative to the processor 710, and these remote memories can be connected to the speech signal analysis and synthesis device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0083] The input device 730 can receive input digital or character information, and generate signals related to user settings and function control of the speech signal analysis and synthesis device. The output device 740 may include a display device such as a display screen.

[0084] The one or more modules are stored in the memory 720, and when executed by the one or more processors 710, they perform the speech signal analysis and synthesis method in any of the above method embodiments.

[0085] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0086] The electronic devices in this application embodiments exist in various forms, including but not limited to:

[0087] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0088] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include: PDAs, MIDs, and UMPCs, etc.

[0089] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players, handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0090] (4) Other airborne electronic devices with data interaction capabilities, such as vehicle-mounted systems installed on vehicles.

[0091] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A speech signal analysis and synthesis method, comprising: The process of obtaining the Mel spectrum and converting it into a source signal using a sine model includes: converting the Mel spectrum into at least one frame of linear spectrum using an inverse Mel filter and obtaining the spectral peaks in the at least one frame of linear spectrum; matching all the spectral peaks to obtain a peak track group; sampling at least one group of the peak track group and calculating the phase of the sampled at least one group of peak tracks using a sine function to obtain the source signal. The source signal is divided into frames, and the framed source signal and the Mel spectrum are input into a neural network model to obtain the output of the neural network model. The framed source signal and the Mel spectrum are at the same frame level. The output is passed through at least a first-order filter bank to obtain the final speech.

2. The method according to claim 1, wherein, The process of matching all spectral peaks to obtain peak orbital groups includes: The spectral peaks of each frame are sorted, and spectral peaks with similar frequencies in each frame are matched.

3. The method according to claim 2, wherein, The method further includes: The matching results of the two peaks in the spectrum matching are retained if the frequency difference and peak energy difference are less than a preset threshold, wherein the matching results are peak orbital groups that are continuous in time, amplitude and frequency.

4. The method according to claim 1, wherein, Sampling at least one of the wave crest orbit groups includes: For each group of peak tracks in the peak track group, the energy and frequency of the spectral peak are sampled using linear interpolation until the sampling rate is consistent with the synthesized speech.

5. The method according to claim 1, wherein, The phase of at least one set of peak orbits after calculation and sampling includes: The phase of the crest orbit at each moment during the duration is calculated based on the preset initial phase, and the end time of the orbit is recalibrated by returning to the preset initial phase with the last phase during the duration.

6. The method according to claim 5, wherein, After calculating the phase of at least one set of sampled peak orbits, the method further includes: The amplitude value of the wave crest track at each moment is calculated by using a sine function, and all the wave crest tracks are superimposed to obtain the source signal.

7. The method according to claim 1, wherein, The step of framing the source signal includes: The source signal is low-pass filtered, and the low-pass filtered source signal is divided into a frame-level sequence that is the same as the Mel spectrum feature frame.

8. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images