A post-processing method for improving chip-end speech synthesis

Abstract

A post-processing method for improving chip-end speech synthesis, comprising the following steps: S1. designing a window function, setting a minimum frequency part window value and a maximum frequency part window value, and calculating a total number of windows and a window value; S2. sequentially taking all feature dimensions of each frame of acoustic features as a vector P, sequentially placing the windows obtained in step S1 on the vector P, calculating the mean value of the feature values, and filling in the gaps; and S3. performing enhancement processing on the acoustic features. The post-processing method for improving chip-end speech synthesis can constrain the calculation range of the mean value of feature values of different frequencies, enhance the recognizability of speech features while suppressing noise by enhancing the spectrum and modifying the spectral distribution of the speech signal, and improve the coherence and synthesis quality of speech synthesis; and the post-processing method requires small computing power and is easy to deploy to the chip end.

Description

Technical Field

[0001] This invention belongs to the field of speech synthesis technology, specifically relating to a post-processing method for improving speech synthesis at the chip level. Background Technology

[0002] Due to limitations in computing power and storage, chip-level speech synthesis cannot be implemented using traditional end-to-end technologies. Therefore, a low-cost, chip-level speech synthesis algorithm solution needs to be designed. Based on front-end design, this approach uses linguistics to design a set of linguistic features that describe the pronunciation rules of the language. A simple DNN model is used to obtain the acoustic features corresponding to the text. These acoustic features are then fed into a traditional vocoder (such as "world") to obtain audio data. This solution requires fewer hardware resources and can be successfully deployed on the chip level, but the speech synthesis quality is affected. Due to the limitations of the DNN model, the generated acoustic features differ significantly from the actual acoustic features. Directly feeding these features into a traditional vocoder can easily produce noise and distortion; therefore, post-processing of the acoustic features is essential.

[0003] There are many ways to post-process acoustic features, which vary in different speech synthesis systems and implementations. Most methods optimize acoustic features by focusing on aspects such as spectral smoothing, noise suppression, and feature correction.

[0004] The open-source framework Merlin (https: / / github.com / CSTR-Edinburgh / merlin) uses the SPTK (https: / / sp-tk.sourceforge.net) toolkit to perform post-processing of acoustic features, which is computationally complex and time-consuming. Therefore, considering the unique characteristics of the acoustic features output by DNN models, it is essential to design an algorithm that can address outlier issues and smoothly enhance acoustic features. Summary of the Invention

[0005] To address the shortcomings of the existing technology, this invention discloses a post-processing method for improving speech synthesis at the chip level.

[0006] The post-processing method for on-chip speech synthesis described in this invention includes the following steps: S1. Design window functions, total number of windows nc= M is the lowest frequency window value, and N is the highest frequency window value. This indicates rounding down, with each window value W representing a floor. i The calculation method is as follows: (1) (2) In formula (1), the lins operation represents nc values ​​obtained by taking values ​​at equal intervals between M and N, and L represents the set of these nc values, L = (L i ), i=0,1,…nc-1; S2. Sequentially take all the feature dimensions of the acoustic features in each frame and set them as vector P. Each element of vector P is the feature dimension of the acoustic features. Perform the following calculation: Place the nc windows obtained in step S1 onto vector P in sequence, with the starting points being P0, P1, ..., P1. nc-1 Calculate the average of all element values ​​in vector P covered by each window, and denote it as... , Then the mean of the eigenvalues ​​corresponding to the j-th element is m. j The calculation formula is as follows: (3) In formula (3), m j P represents the mean of the eigenvalues ​​obtained in the current window. k This represents all feature dimensions covered by the window; Then the mean of the eigenvalues ​​at the j-th element position is ave j =m j And fill in the missing eigenvalues ​​at the positions not calculated in formula (3); S3. Enhance the acoustic features as follows: S31. Define the gain factor at zero frequency as zl and the gain factor at maximum frequency as zu. Then, the vector representing the gain factor can be obtained by the following formula: (5) The lins operation represents taking n values ​​at equal intervals between zl and zu, where B is a gain factor vector of length n, and n is the length of the feature dimension. B = (B i ), i=1,2…n-2; B i Let be the gain factor at the i-th position; S32. Obtain the data-augmented feature values ​​using the following formula: (6) In formula (6) P represents the feature dimension obtained after processing. i Represents all feature dimensions of the acoustic features before processing, i=0,1,…n-1; the final combination of all The processed acoustic characteristics are obtained.

[0007] Preferably, the lowest frequency portion window value M is set to 9, and the highest frequency portion window value N is set to 12.

[0008] Preferably, the gain factor zl at zero frequency is 2, and the gain factor zu at maximum frequency is 1.6.

[0009] Preferably, the acoustic features in step S2 are acoustic features output by a DNN model based on a speech synthesis scheme.

[0010] Preferably, the feature value mean completion in step 2 specifically involves: The mean of the eigenvalues ​​at the j-th element position ave j (4) In formula (4), ave j This represents the mean of the eigenvalues ​​at position j. This represents the mean eigenvalues ​​corresponding to the l-th and u-th elements calculated according to formula (3).

[0011] The post-processing method for improving speech synthesis at the chip level described in this invention can constrain the calculation range of the mean value of feature values ​​at different frequencies. By enhancing the spectrum and modifying the spectrum distribution of the speech signal, it can enhance the recognizability of speech features while suppressing noise, thereby improving the coherence and synthesis quality of speech synthesis. Furthermore, the post-processing method described in this invention requires little computing power and is easy to deploy at the chip level. Attached Figure Description

[0012] Figure 1 The diagram shown is a schematic representation of a specific implementation of the post-processing method described in this invention. Figure 2 The diagram shown is a specific embodiment of step S2 in the post-processing method of the present invention. Figure 3 The image shown is a spectrum diagram of a specific embodiment of the present invention. Detailed Implementation

[0013] To more intuitively and clearly describe the specific details of the technical solution of the present invention, a detailed description will be provided below in conjunction with specific embodiments and example drawings.

[0014] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0015] like Figure 1As shown, the acoustic features output by the acoustic model of the present invention are continuous multi-frame data. It is assumed that the acoustic feature matrix output by the acoustic model is an [m,n] order matrix, where m represents the number of acoustic feature frames and n represents the length of the feature dimension of each frame. The low-dimensional part of the feature dimension is the low-frequency part feature, and the high-dimensional part is the high-frequency part feature.

[0016] For each frame, perform the following processing: S1. Design window functions without feature expansion. The total window size is nc = M is the lowest frequency window value, and N is the highest frequency window value. (Symbols omitted). This indicates rounding down, with the total number of windows denoted as nc. The window values ​​change smoothly from high frequency to low frequency, and each window value W... i The calculation method is as follows: (1) (2) In formula (1), the lins operation represents nc values ​​obtained by taking values ​​at equal intervals between M and N, and L represents the set of these nc values, L = (L i ), i=0,1,…nc-1.

[0017] In formula (2) This represents rounding down to the nearest integer, which yields nc odd integer values ​​W. i .

[0018] This step yields a set of continuously varying window values, which can constrain the calculation range of the mean value of feature values ​​at different frequencies.

[0019] Low and mid-frequency sounds are the main components of human speech, allowing for clear speech. For signals in this frequency band, especially in longer vocal tracts (such as the larynx), sound waves encounter reflection and interference effects as they pass through the vocal tract. These effects cause resonance enhancement of sound waves in specific frequency regions, forming resonance peaks. Because low and mid-frequency signals have longer wavelengths, sound waves need to travel a longer distance to reach the vocal tract, making resonance peaks easier to form. Therefore, the window value in the low and mid-frequency band should not be set too large, as a large window value will affect the mean value at the resonance peak, thus affecting the clarity and audibility of the synthesized sound. Increasing the window value at high frequencies can smooth out the synthesized sound and reduce the sound of "sharp" parts. The window values ​​obtained from formula (2) are a set of odd values, which can more accurately calculate the characteristic mean value within its coverage area.

[0020] S2. Sequentially extract all feature dimensions of the acoustic features for each frame and set them as vector P. The dimension of vector P is the length n of the feature dimension, and each element of vector P is the feature dimension of the acoustic feature. Perform the following calculation: Place the nc windows obtained in step S1 onto vector P sequentially, calculate the average value of all element values ​​covered by each window on vector P, and record the average value. , The calculation formula is as follows: (3) In formula (3), m j P represents the mean of the eigenvalues ​​obtained in the current window. k This represents all feature dimensions covered by the window.

[0021] The purpose of calculating the mean in formula (3) is to achieve speech enhancement at a specific location. If the mean of the feature location differs significantly from the feature value of that location, it indicates that the location is a peak location. After performing spectral enhancement on that location using formula (6), the features of the peak location can be made clearer and more obvious.

[0022] Then, for each element position of vector P, the mean of the eigenvalues ​​at that position is... (4) In formula (4), ave j The mean of the eigenvalues ​​at the j-th element position is represented by the mean vector AVE = ( ave j j=0,1…n-1.

[0023] Since formula (3) does not cover all elements of vector P, formula (4) compensates for the mean values ​​at the positions of elements not covered by formula (3). For each position less than l, the mean eigenvalue is equal to... For each position greater than nu, the mean eigenvalue is equal to Formula (4) ensures the continuity of the mean and avoids sudden jumps at the beginning and end of the value. The beginning and end features can still participate in the calculation when performing spectral enhancement.

[0024] S3. Enhance the acoustic features as follows: S31. Define the gain factor at zero frequency as zl and the gain factor at maximum frequency as zu. Then, the vector representing the gain factor can be obtained by the following formula: (5) The lins operation represents taking n values ​​at equal intervals between zl and zu, where B is a gain factor vector of length n, and n is the length of the feature dimension. B = (B i ), i=1,2…n-2; B iLet be the gain factor at the i-th position.

[0025] S32. Obtain the data-augmented feature values ​​using the following formula: (6) In formula (6) P represents the feature dimension obtained after processing. i Represents all feature dimensions of the acoustic features before processing, i=0,1,…n-1.

[0026] This step enhances the recognizability of speech features by spectral enhancement and modifying the spectral distribution of the speech signal, while suppressing noise and improving the coherence and quality of speech synthesis.

[0027] The post-processing method for improving on-chip speech synthesis described in this invention enhances the recognizability of speech features by spectral enhancement and modification of the spectral distribution of the speech signal, while suppressing noise, thus greatly improving the coherence and quality of speech synthesis. The post-processing method proposed in this invention requires low computational power and is easily deployed on the chip. Specific Implementation

[0028] Assume that the acoustic features output by the DNN (Deep Neural Networks) speech synthesis scheme upon which this invention is based are a matrix of size [100, 60]. 100 represents the number of acoustic feature frames, 60 represents the length of the feature dimension of each frame, and the corresponding sampling rate is 16000.

[0029] S1. Design a window function, setting the window value M for the lowest frequency part to 9 and the window value N for the highest frequency part to 12; in this embodiment, the values ​​of M and N are empirical values ​​when the sampling rate is 16000, and they will be adjusted according to the sampling rate in actual applications.

[0030] This allows us to determine the number of window values. The value is 50. The window value W for 50 windows can be obtained from formula (2). i .

[0031] (2) S2. Sequentially extract the acoustic features for each frame. In this embodiment, there are a total of 100 frames. For formula (5), take the empirical values ​​zl=2 and zu=1.6 to obtain a set of 60-dimensional vectors. Perform the following calculations for each frame: The current frame feature is labeled as vector P, which is a 60-dimensional vector; for example... Figure 2 The figure shows a representation of vector P, P0, P1…P 59 These are the 60 elements of vector P.

[0032] Slide 50 window values ​​across the feature vector P, calculate the mean of the features covered by the corresponding window values, and use formula (3) to calculate the mean at each position, such as... Figure 2 As shown, according to formula (2), the window value W1=7 of the first window, the starting point is the first element of the feature vector P, and the position corresponding to the midpoint of the first window is P3. Then, according to formula (3), the mean value of the feature value corresponding to the position P3 is calculated.

[0033] If all 50 windows are used, the starting point of the 50th window is the 49th element of the feature vector P, and the window value W is... 50 =11, the position corresponding to the midpoint of the 50th window is P. 55 Then P is calculated according to formula (3). 55 The mean of the feature values ​​corresponding to the positions, and for the three front points P0, P1, and P2 that are not covered by formula (3), and the back point P 56 P 57 P 58 P 59 The mean eigenvalues ​​corresponding to the four points are supplemented with marginal mean features according to formula (4) to obtain the complete mean vector AVE.

[0034] S3. Enhance the acoustic features and calculate the enhanced acoustic features according to formula (6), which is the feature obtained by post-processing of the scheme described in this invention.

[0035] Figure 3 To utilize the method described in this invention to obtain the acoustic feature map corresponding to the text "Tianjin TV", from... Figure 3 It is evident that the acoustic characteristics before processing are blurred, and the synthesized audio is low and hoarse. After processing, the high and low frequency characteristics are more obvious, and the sound is crisper and clearer.

[0036] The foregoing descriptions are preferred embodiments of the present invention. Unless there is a clear contradiction between the preferred embodiments or a prerequisite for a particular preferred embodiment, the preferred embodiments can be arbitrarily combined and used. The embodiments and specific parameters described are only for clearly illustrating the inventor's invention verification process and are not intended to limit the scope of patent protection of the present invention. The scope of patent protection of the present invention shall still be determined by its claims. Similarly, any equivalent structural changes made based on the description and drawings of the present invention shall also be included within the scope of protection of the present invention.

Claims

1. A post-processing method for improving on-chip speech synthesis, characterized in that, Includes the following steps: S1. Design window functions, total number of windows nc= M is the lowest frequency window value, and N is the highest frequency window value. This indicates rounding down, with each window value W representing a floor. i The calculation method is as follows: (1) (2) In formula (1), the lins operation represents nc values ​​obtained by taking values ​​at equal intervals between M and N, and L represents the set of these nc values, L = (L i ), i = 0, 1, ..., nc-1; S2. Sequentially take all the feature dimensions of the acoustic features in each frame and set them as vector P. Each element of vector P is the feature dimension of the acoustic features. Perform the following calculation: Place the nc windows obtained in step S1 onto vector P sequentially, with the starting points being P0, P1…P… nc-1 Calculate the average of all element values ​​in vector P covered by each window, and denote it as... , Then the mean of the eigenvalues ​​corresponding to the j-th element is m. j The calculation formula is as follows: （3） In formula (3), m j P represents the mean of the eigenvalues ​​obtained in the current window. k This represents all feature dimensions covered by the window; Then the mean of the eigenvalues ​​at the j-th element position ave j =m j ; And fill in the missing eigenvalues ​​at the positions not calculated in formula (3); S3. Enhance the acoustic features as follows: S31. Define the gain factor at zero frequency as zl and the gain factor at maximum frequency as zu. Then, the vector representing the gain factor can be obtained by the following formula: （5) The lins operation represents taking n values ​​at equal intervals between zl and zu, where B is a gain factor vector of length n, and n is the length of the feature dimension. B = (B i ), i=1,2…n-2; B i Let be the gain factor at the i-th position; S32. Obtain the data-augmented feature values ​​using the following formula: （6） In formula (6) P represents the feature dimension obtained after processing. i Represents all feature dimensions of the acoustic features before processing, i=0,1,…n-1; the final combination of all The processed acoustic characteristics are obtained.

2. The post-processing method as described in claim 1, characterized in that, The lowest frequency window value M is set to 9, and the highest frequency window value N is set to 12.

3. The post-processing method as described in claim 1, characterized in that, The gain factor at zero frequency is zl=2, and the gain factor at maximum frequency is zu=1.

6.

4. The post-processing method as described in claim 1, characterized in that, The acoustic features in step S2 are acoustic features output by the DNN model of the speech synthesis scheme.

5. The post-processing method as described in claim 1, characterized in that, The feature value mean completion in step 2 is specifically as follows: The mean of the eigenvalues ​​at the j-th element position ave j （4） In formula (4), ave j This represents the mean of the eigenvalues ​​at position j. This represents the mean eigenvalues ​​corresponding to the l-th and u-th elements calculated according to formula (3).

Images