8results about How to "Improve speech synthesis" patented technology

The invention relates to artificial intelligence, and discloses a speech synthesis method and device with a conditional matching stream, equipment and a medium, and the method comprises the steps: receiving text information, coding the text information into a high-dimensional implicit vector, and obtaining a text information vector; label information is formed according to the alignment relation between the text information and the voice, and the relation between the acoustic features of the text information and the voice is predicted; performing matching in the voice data set according to the text information vector to obtain corresponding text-related voice, generating intermediate-state voice according to Gaussian noise, the text-related voice and the time step, performing noise prediction according to the intermediate-state voice and the time step by adopting a conditional matching flow model to obtain predicted noise, and outputting the predicted noise. Calculating the loss between the input Gaussian noise and the prediction noise, and combining the prediction noise with the intermediate state voice to extract and obtain the acoustic features of the text related voice; and generating a decoded speech waveform according to the obtained acoustic characteristics of the text-related speech by adopting an acoustic decoder. And the speech synthesis quality is improved.

This application discloses a speech synthesis method, apparatus, device, and storage medium. The method involves analyzing the original text to be synthesized to obtain a phoneme sequence; inputting the phoneme sequence into a configured speech synthesis model to obtain synthesized speech output by the model. The speech synthesis model is a final speech synthesis model after parameter adjustment of the basic speech synthesis model, using the scoring results of multiple candidate speech samples corresponding to the input test text synthesized by the basic speech synthesis model as reward signals. The scoring results of each candidate speech sample conform to the user's auditory perception goals. This application adds user auditory feedback signals (i.e., the scoring results as reward signals) to the training process of the speech synthesis model, guiding the speech synthesis model to optimize model parameters in a direction that better conforms to the user's auditory perception, making the synthesized speech more in line with the user's auditory perception goals and improving the speech synthesis effect.

The present disclosure relates to a speech synthesis method, device, medium and electronic device, including: obtaining text features corresponding to a text to be synthesized; inputting the text features into a pre-trained speech synthesis model to obtain a target speech synthesized according to the text to be synthesized; wherein the speech synthesis model synthesizes the target speech by directly converting the text features into speech waveform points corresponding to the target speech. In this way, when synthesizing speech according to text, there is no need to predict any intermediate features, which not only reduces the size of the speech synthesis model, making it more convenient to deploy it in various terminals for offline use, but also because there are no intermediate features, it can reduce the information loss in the middle layer of the model, and reduce the information difference between the synthesized target speech and the text features, thereby also improving the speech synthesis effect of the model to a certain extent, and further improving the user experience of offline speech synthesis.

This invention, based on decoding research in speech brain-computer interfaces, discloses a discrete decoding method guided by continuous decoding in speech brain-computer interfaces. This method utilizes a deep learning model where continuous and discrete decoding systems coexist. First, it trains the continuous decoding system on information such as Mel spectrum, fundamental frequency, and non-periodic signals from the acoustic data. Then, it uses the features extracted from the continuous decoding to guide the discrete decoding system through transfer learning, ultimately completing the training of the discrete decoding system (i.e., the classification task). The performance of this discrete decoding is significantly higher than that of simple discrete decoding. Furthermore, the fundamental frequency and other signals output by the trained continuous decoding system can be used as adjustment inputs for subsequent speech synthesis systems based on the discrete decoding results, enhancing the speech synthesis effect of the discrete decoding results.

The invention provides a speech synthesis model training method, a speech synthesis method, a speech synthesis device and a computer readable storage medium. The speech synthesis model training method comprises the following steps: inputting a speech text into a speech synthesis model to be trained, and obtaining predicted acoustic features; obtaining a spectrum similarity loss value based on the actual spectrum energy high-density subspace matrix and the predicted spectrum energy high-density subspace matrix; obtaining an energy distribution loss value based on an actual spectrum energy low-density subspace matrix and the predicted spectrum energy low-density subspace matrix; and training the speech synthesis model according to the spectrum similarity loss value and the energy distribution loss value. Through the above mode, the speech synthesis device trains the speech synthesis model through the frequency spectrum similarity loss function and the energy distribution loss function, so as to alleviate the problems of overall plain and local flaws of the synthesized speech, and improve the speech synthesis effect.

The invention relates to the field of artificial intelligence and financial science and technology, and discloses a speech synthesis method, a training method, a device, equipment and a storage medium, and the method comprises the steps: inputting a current text into a trained emotion intensity model; based on the trained emotion intensity model, obtaining identification information of the current text; obtaining the ranking score of the current emotion category and the current emotion intensity in the identification information; performing feature extraction on the current emotion category to obtain a first feature vector, performing feature extraction on the ranking score of the current emotion intensity to obtain a second feature vector, and performing feature extraction on the current text to obtain a third feature vector; performing fusion processing on the first feature vector, the second feature vector and the third feature vector to determine a fourth feature vector; and performing speech synthesis processing on the fourth feature vector to generate a synthesized speech conforming to the current emotion category and the current emotion intensity. The speech synthesis effect of the current text can be improved.

The invention discloses a phoneme sequence determination method and device, computer equipment and a storage medium. The method comprises the following steps: inputting a target text into a first model to obtain a candidate phoneme sequence; generating a first coding feature according to words in the target text, and generating a second coding feature according to phonemes in the candidate phoneme sequence; and inputting the first coding feature and the second coding feature into a second model to obtain a target similarity, and determining a target phoneme sequence of the target text from the candidate phoneme sequences according to the target similarity. According to the embodiment of the invention, the generation relation from the text to the phoneme sequence is directly established through the preset first model, the effect of the generated candidate phoneme sequence can be ensured, the phoneme sequence is screened by using the similarity, the accuracy of the generated phoneme sequence is further ensured, and thus the effect of subsequent speech synthesis is improved.

The invention relates to the technical field of speech synthesis, and discloses a speech synthesis method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining reference speech data and a to-be-synthesized text; encoding the reference voice data through a preset rhythm encoder to obtain a rhythm embedding vector; performing feature extraction processing on the reference voice data through a preset voice recognition model to obtain a compensation feature vector; encoding the compensation feature vector through a preset rhythm compensation encoder to obtain a rhythm compensation vector; fusing the rhythm embedding vector and the rhythm compensation vector to obtain a rhythm feature vector; and synthesizing the to-be-synthesized text and the rhythm feature vector through a preset speech synthesis model to generate a target synthesized speech. According to the method, the voice synthesis is assisted through rhythm embedding and rhythm compensation, so that the loss of rhythm information is reduced, the voice synthesis effect is improved, intelligent and humanized services are embodied in a financial service scene, and the user experience is improved.