Speech speed adjustment method and device, electronic equipment and readable storage medium

Abstract

The present disclosure relates to a speech speed adjustment method and device, electronic equipment and readable storage medium, wherein the method comprises: obtaining a text to be synthesized, inputting the text to be synthesized into a speech synthesis model, the speech synthesis model comprising an encoding network, an attention network and a decoding network, the encoding network converting the input text to be synthesized into an acoustic feature sequence; the attention network is used to output an attention vector, and the decoding network is used to output a target spectrum corresponding to the text to be synthesized according to the attention vector, the acoustic feature sequence and a state transition control factor; and obtaining a target audio with a target speech speed through the target spectrum corresponding to the text to be synthesized. By introducing the state transition control factor into the speech synthesis model, the number of target spectrums corresponding to the text to be synthesized is dynamically controlled by using the state transition control factor, and flexible speech speed adjustment is realized in the speech synthesis process. And the synthesized audio has high naturalness of sound, which is conducive to improving the user experience.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of Internet, and particularly relates to a speech speed adjustment method and device, electronic equipment and readable storage medium. BACKGROUND

[0002] Electronic equipment often needs to adjust the speech speed of audio. For example, when a user uses electronic equipment to play a video, the user often plays the video at a speed of 1.25 times, 1.5 times, 2.0 times, etc., and thus needs to adjust the speech speed of the audio in the video to adapt to the speed of the video.

[0003] In the prior art, the speech speed adjustment of audio is usually achieved by using digital signal processing (DSP) technology to discard, resample, interpolate, etc. the audio, thereby lengthening or shortening the time length of the audio. SUMMARY

[0004] To solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a speech speed adjustment method, device, electronic equipment and readable storage medium.

[0005] In a first aspect, the present disclosure provides a speech speed adjustment method, comprising:

[0006] obtaining a to-be-synthesized text;

[0007] inputting the to-be-synthesized text into a speech synthesis model to obtain a target spectrum corresponding to the to-be-synthesized text output by the speech synthesis model; wherein the speech synthesis model comprises an encoding network, an attention network and a decoding network, the encoding network is configured to convert the input to-be-synthesized text into an acoustic feature sequence; the attention network is configured to output an attention vector, and the decoding network is configured to output the target spectrum corresponding to the to-be-synthesized text according to the input attention vector, the acoustic feature sequence and a state transition control factor; and the state transition control factor is configured to control the number of target spectrums corresponding to the to-be-synthesized text;

[0008] obtaining a target audio according to the target spectrum corresponding to the to-be-synthesized text, wherein the target audio has a target speech speed.

[0009] As a possible implementation, if the state transition control factor is less than a preset threshold, the target speech speed of the target audio is less than a reference speech speed.

[0010] If the state transition control factor is greater than the preset threshold, the target speech speed of the target audio is greater than the reference speech speed.

[0011] If the state transition control factor is equal to the preset threshold value, a target speech rate of the target audio is equal to the reference speech rate.

[0012] As a possible implementation, the decoding network comprises a first fully connected layer, a second fully connected layer, a linear layer, and a recurrent neural network layer.

[0013] The decoding network is configured to output a target spectrum corresponding to the text to be synthesized according to the input attention vector, the sequence of acoustic features, and a state transition control factor, comprising:

[0014] The attention vector and the sequence of acoustic features are weighted to obtain a target vector of a current step, and the target vector of the current step is input into the recurrent neural network layer.

[0015] The recurrent neural network layer obtains a target state quantity of the current step according to the target vector of the current step, a state transition control factor, and a target state quantity of a previous step.

[0016] The target state quantity of the current step is input into the first fully connected layer to obtain a target spectrum of the current step output by the first fully connected layer, and the target state quantity of the current step is input into the second fully connected layer to obtain a stop symbol output by the second fully connected layer.

[0017] If the stop symbol indicates that the end position of the text to be synthesized has not been reached, the target spectrum of the current step is extracted through the linear layer and input into the attention network, so that the attention network updates the attention vector.

[0018] The process of weighting the attention vector and the sequence of acoustic features to obtain the target vector of the current step, and inputting the target vector of the current step into the recurrent neural network layer, the recurrent neural network layer obtaining the target state quantity of the current step according to the target vector of the current step, a state transition control factor, and a target state quantity of a previous step, and inputting the target state quantity of the current step into the first fully connected layer to obtain the target spectrum of the current step output by the first fully connected layer, and inputting the target state quantity of the current step into the second fully connected layer to obtain a stop symbol output by the second fully connected layer, is repeated until the stop symbol indicates that the end position of the text to be synthesized has been reached.

[0019] As a possible implementation, the obtaining of the target state quantity of the current step according to the target vector of the current step, a state transition control factor, and a target state quantity of a previous step comprises:

[0020] According to the target vector of the current step, the state transition control factor, and the target state quantity of the previous step, mask weighted fusion is performed to obtain the target state quantity of the current step.

[0021] As a possible implementation, the mask weighted fusion according to the target vector of the current step, the state transition control factor, and the target state quantity of the previous step to obtain the target state quantity of the current step includes:

[0022] According to the target vector, an initial state quantity of the current step is obtained.

[0023] According to the state transition control factor, a mask is generated, and the initial state quantity of the current step and the target state quantity of the previous step are weighted fused according to the mask to obtain the target state quantity of the current step.

[0024] As a possible implementation, before the target state quantity of the current step is obtained according to the target vector of the current step, the state transition control factor, and the target state quantity of the previous step, the method further includes:

[0025] The size of the state transition control factor corresponding to the current step is updated.

[0026] As a possible implementation, the size of the state transition control factor corresponding to the current step is updated, including:

[0027] According to one or more of the target speech rate, the acoustic feature sequence corresponding to the text to be synthesized, and the importance of the text content corresponding to the current step, the size of the state transition control factor corresponding to the current step is updated.

[0028] In a second aspect, the present disclosure provides a speech rate adjustment device, including:

[0029] An acquisition module is configured to acquire text to be synthesized.

[0030] A spectrum feature extraction module is configured to input the text to be synthesized into a speech synthesis model to obtain a target spectrum corresponding to the text to be synthesized output by the speech synthesis model. The speech synthesis model includes an encoding network, an attention network, and a decoding network. The encoding network is configured to convert the input text to be synthesized into an acoustic feature sequence. The attention network is configured to output an attention vector. The decoding network is configured to output the target spectrum corresponding to the text to be synthesized according to the input attention vector, acoustic feature sequence, and state transition control factor. The state transition control factor is configured to control the number of target spectrums.

[0031] An audio processing module is configured to obtain a target audio according to the target spectrum corresponding to the text to be synthesized. The target audio has a target speech rate.

[0032] As a possible implementation, if the state transition control factor is less than a preset threshold, a target speech rate of the target audio is less than a reference speech rate; if the state transition control factor is greater than the preset threshold, the target speech rate of the target audio is greater than the reference speech rate; if the state transition control factor is equal to the preset threshold, the target speech rate of the target audio is equal to the reference speech rate.

[0033] As a possible implementation, the decoding network comprises a first full connection layer, a second full connection layer, a linear layer and a recurrent neural network layer.

[0034] The spectrum feature extraction module is specifically configured to perform weighted calculation on the attention vector and the acoustic feature sequence to obtain a target vector, and input the target vector into the first full connection layer and the second full connection layer through the recurrent neural network.

[0035] The first full connection layer obtains a target spectrum of a current step according to the input target vector, a state transition control factor and an existing target spectrum; and the second full connection layer obtains a stop symbol according to the input target vector.

[0036] If it is determined according to the stop symbol that a preset stop condition is not met, the target spectrum of the current step is input into the attention network through the linear layer and the recurrent neural network layer, so that the attention network updates the attention vector.

[0037] The attention vector and the acoustic feature sequence are weighted and calculated to obtain a target vector, and the target vector is input into the first full connection layer and the second full connection layer through the recurrent neural network, so that the first full connection layer obtains the target spectrum of the current step according to the input target vector, a state transition control factor and an existing target spectrum; and the second full connection layer obtains the stop symbol according to the input target vector, until it is determined according to the stop symbol that the stop condition is met.

[0038] As a possible implementation, the spectrum feature extraction module is specifically configured to obtain an initial spectrum of a current step according to the target vector; and perform mask weighted fusion on the initial spectrum of the current step and the existing target spectrum according to the state transition control factor to obtain the target spectrum of the current step.

[0039] As a possible implementation, the spectrum feature extraction module is specifically configured to mask and weight fuse the initial spectrum of the current step and the target spectrum of the previous step according to the state transition control factor to obtain the target spectrum of the current step; and the existing target spectrum includes the target spectrum of the previous step.

[0040] As a possible implementation, the spectrum feature extraction module is further configured to update the size of the state transition control factor corresponding to the current step.

[0041] As a possible implementation, the spectrum feature extraction module is specifically configured to update the size of the state transition control factor corresponding to the current step according to one or more of the target speech rate, the acoustic feature sequence corresponding to the text to be synthesized, and the importance of the text content of the current step.

[0042] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor;

[0043] The memory is configured to store computer program instructions;

[0044] The processor is configured to execute the computer program instructions, so that the electronic device implements the speech rate adjustment method according to any one of the first aspect.

[0045] In a fourth aspect, the present disclosure provides a readable storage medium, comprising: computer program instructions; when at least one processor of an electronic device executes the computer program instructions, the electronic device implements the speech rate adjustment method according to any one of the first aspect.

[0046] In a fifth aspect, the present disclosure provides a program product, comprising: computer program instructions; the computer program instructions are stored in a readable storage medium, and an electronic device obtains the computer program instructions from the readable storage medium; when at least one processor of the electronic device executes the computer program instructions, the electronic device implements the speech rate adjustment method according to any one of the first aspect.

[0047] This disclosure provides a speech rate adjustment method, apparatus, electronic device, and readable storage medium. The method includes: acquiring text to be synthesized; inputting the text to be synthesized into a speech synthesis model, the speech synthesis model including an encoding network, an attention network, and a decoding network; wherein the encoding network converts the input text to be synthesized into an acoustic feature sequence; the attention network outputs an attention vector; and the decoding network outputs a target spectrum corresponding to the text to be synthesized based on the attention vector, the acoustic feature sequence, and a state transition control factor; and then obtaining a target audio with a target speech rate using the target spectrum corresponding to the text to be synthesized. This disclosure, by introducing a state transition control factor into the speech synthesis model, dynamically controls the number of target spectra corresponding to the text to be synthesized, thereby achieving flexible speech rate adjustment during speech synthesis. Furthermore, the audio synthesized using the method provided by this disclosure has a high degree of naturalness, which is beneficial for improving user experience. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0049] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart of a speech rate adjustment method provided in an embodiment of this disclosure;

[0051] Figure 2 This is a schematic diagram of the structure of a speech synthesis model provided in an embodiment of the present disclosure;

[0052] Figure 3 This is a schematic diagram of the structure of a speech synthesis model provided in another embodiment of the present disclosure;

[0053] Figure 4 This is a schematic diagram of the speech rate adjustment device provided in one embodiment of the present disclosure;

[0054] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0055] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0056] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. The present disclosure can, however, be practiced without the specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the present disclosure.

[0057] When adjusting the speech rate of audio by using DSP technology, since the DSP technology can only adjust the rate of the entire audio, it is unable to flexibly adjust the speech rate at different time, which may result in that the speech rate of some audio segments in the entire audio is not suitable for the audio segments.

[0058] In addition, when adjusting the speech rate by using the DSP technology, the effective spectrum is adjusted, which may result in that the pitch of the audio after adjusting the speech rate is changed, that is, the timbre is changed, thereby resulting in that the naturalness of the sound is low.

[0059] Therefore, the present disclosure provides a speech rate adjustment method and device, an electronic device, a readable storage medium and a computer program product, wherein the method introduces a state transition control factor in a speech synthesis model, controls the number of target spectrums corresponding to the to-be-synthesized text output by the speech synthesis model by using the state transition control factor, and flexibly adjusts the speech rate in the speech synthesis process. The audio synthesized by the method provided by the present disclosure has high naturalness of sound, which is beneficial to improving the user experience.

[0060] The speech rate adjustment method provided by the present disclosure can be executed by an electronic device. Exemplarily, the electronic device can include, but is not limited to, a tablet computer, a mobile phone (such as a folding screen mobile phone, a large-screen mobile phone, etc.), a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a smart television, a smart screen, a high-definition television, a 4K television, a smart speaker, a smart projector, and the like, and the present disclosure does not limit the specific type of the electronic device.

[0061] In the following embodiments, the speech rate adjustment method executed by the electronic device is taken as an example, and the speech rate adjustment method provided by the present disclosure is described in detail in combination with the accompanying drawings and application scenarios.

[0062] Please refer to Figure 1 , Figure 1 The flowchart of the speech rate adjustment method provided by an embodiment of the present disclosure is shown in FIG. 1. As shown in FIG. 1, the speech rate adjustment method provided by the present disclosure includes the following steps. Figure 1As shown, the speech speed adjustment method provided by the embodiment can include:

[0063] S101, obtaining a to-be-synthesized text.

[0064] The electronic device can obtain a to-be-synthesized text for synthesizing target audio, and the to-be-synthesized text includes elements for synthesizing audio. Wherein, the to-be-synthesized text can include characters for synthesizing audio, or the to-be-synthesized text can include phonemes for synthesizing audio.

[0065] The disclosure does not limit the acquisition method of the to-be-synthesized text, for example, the to-be-synthesized text can be input by the user, or the to-be-synthesized text can be obtained by the electronic device through audio recognition, translation and the like. The disclosure does not limit the language category of the to-be-synthesized text, which can be Chinese, English, or other language categories. In addition, the disclosure does not limit the number of elements (i.e. the length of the text) included in the to-be-synthesized text, the text content and other parameters.

[0066] S102, inputting the to-be-synthesized text into a speech synthesis model to obtain a target spectrum corresponding to the to-be-synthesized text output by the speech synthesis model.

[0067] The speech synthesis model is a pre-trained machine learning model capable of speech synthesis, wherein the speech synthesis model can also control the speech speed of the synthesized audio during speech synthesis. The disclosure does not limit the type and network structure of the speech synthesis model.

[0068] As a possible implementation, refer to Figure 2 As shown in the embodiment, the speech synthesis model 10 can include an encoding network 11, a decoding network 12, and an attention network 13, wherein the attention network 13 is arranged between the encoding network 11 and the decoding network 12.

[0069] Specifically, the encoding network 11 receives the to-be-synthesized text as input, and can obtain acoustic features corresponding to each element by analyzing acoustic feature sequences of different acoustic dimensions of each element in the to-be-synthesized text, wherein the acoustic features corresponding to each element form an acoustic feature sequence corresponding to the to-be-synthesized text in the order of each element.

[0070] The above different acoustic dimensions can include but are not limited to one or more of the following dimensions: pitch dimension, pause dimension, correlation between phonemes, word boundary dimension, etc. The disclosure does not limit the structure of the encoding network 11 and the implementation of converting the to-be-synthesized text into the acoustic feature sequence.

[0071] It should be noted that if the text to be synthesized includes characters for synthesizing audio, the characters can be converted into phonemes first, and the speech synthesis model performs acoustic feature sequence analysis on each phoneme. The conversion of characters to phonemes can be performed by the speech synthesis model or by other modules independent of the speech synthesis model, and the present disclosure does not limit this. If the text to be synthesized includes phonemes for synthesizing audio, the encoding network 11 can directly perform acoustic feature sequence analysis on each phoneme.

[0072] It should also be noted that when performing acoustic feature sequence analysis on each element included in the text to be synthesized, each phoneme can be analyzed by analyzing a plurality of phonemes located before and after the phoneme, and therefore the acoustic feature sequence information corresponding to the phoneme obtained is able to reflect the context information between the phonemes before and after.

[0073] The decoding network 12 is capable of outputting the target spectrum corresponding to the text to be synthesized according to the attention vector input by the attention network 13, the acoustic feature sequence output by the encoding network 11, and the state transition control factor. The state transition control factor is used to control the number of target spectrums corresponding to the text to be synthesized.

[0074] That is, in the present scheme, the electronic device can use a speech synthesis model with an attention mechanism, control the number of target spectrums corresponding to the text to be synthesized output by the speech synthesis model based on the acoustic feature sequence and the state transition control factor, and thus flexibly control the speed of the synthesized target audio. The more the number of spectrums, the slower the speed of the audio; the less the number of spectrums, the faster the speed of the audio.

[0075] Alternatively, the target spectrum can include any one or more types of spectrums such as a mel spectrum, a combination of BFCC and pitch information, or a spectral envelope.

[0076] As a possible implementation, in the process of speech synthesis, the state transition control factor can be dynamically changed to adapt to the speed requirements of different sentences in the text to be synthesized. That is, in the process of speech synthesis, the size of the state transition control factor is dynamically adjusted through a preset state transition control factor update strategy, thereby controlling the pronunciation duration of different audio segments, so as to achieve the speed of some audio segments or audio positions in the speech synthesis process according to the requirements of the corresponding speed to speed up or slow down.

[0077] The preset state transition control factor updating strategy is not limited in the specific implementation manner. For example, the preset state transition control factor updating strategy can be associated with one or more of the following factors: target speech rate (which can also be understood as a speech rate adjustment ratio, or the difference between the target audio speech rate and the standard speech rate), acoustic feature sequence corresponding to the text to be synthesized, importance of the text content to be expressed in the current step, duration of the sentence (or paragraph) to which the current step belongs, and the like.

[0078] For example, if the target speech rate is fast, the size of the state transition control factor can be reduced; if the target speech rate is slow, the size of the state transition control factor can be increased.

[0079] For example, if the acoustic feature sequence corresponding to the text to be synthesized is analyzed, it is determined that the acoustic feature sequence information of the current step has a stronger correlation with the acoustic feature sequence before the current step, the size of the state transition control factor for the current step can be increased; if the acoustic feature sequence corresponding to the text to be synthesized is analyzed, it is determined that the acoustic feature sequence information of the current step has a stronger correlation with the acoustic feature sequence after the current step, the size of the state transition control factor for the current step can be reduced.

[0080] For example, if the importance of the text content to be expressed in the current step is high, the size of the state transition control factor for the current step can be reduced; if the importance of the text content to be expressed in the current step is low, the size of the state transition control factor for the current step can be increased.

[0081] For example, if the duration of the sentence to which the current step belongs is long but the text content is less, the speech rate at this point can be slowed down, and thus the size of the state transition control factor can be increased; if the duration of the sentence to which the current step belongs is short but the text content is more, the speech rate at this point can be increased, and thus the size of the state transition control factor can be reduced.

[0082] S103, obtaining a target audio according to the target spectrum corresponding to the text to be synthesized, the target audio having a target speech rate.

[0083] The electronic device can play the target audio having the target speech rate according to the target spectrum corresponding to the text to be synthesized at a preset playing rate.

[0084] The method of this embodiment obtains the text to be synthesized and inputs it into a speech synthesis model. The speech synthesis model includes an encoding network, an attention network, and a decoding network. The encoding network converts the input text into an acoustic feature sequence; the attention network outputs an attention vector; and the decoding network outputs the target spectrum corresponding to the text to be synthesized based on the attention vector, the acoustic feature sequence, and a state transition control factor. Then, using the target spectrum corresponding to the text to be synthesized, a target audio with a target speech rate is obtained. This disclosure introduces a state transition control factor into the speech synthesis model, dynamically controlling the number of target spectra corresponding to the text to be synthesized, thereby achieving flexible speech rate adjustment during speech synthesis. Furthermore, the audio synthesized by the method provided in this disclosure has a high degree of naturalness, which is beneficial for improving the user experience.

[0085] Next, the implementation of the speech synthesis model and decoding network provided in this disclosure will be described in detail by way of example.

[0086] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a speech synthesis model provided in one embodiment of this disclosure. Figure 2 Based on the illustrated embodiment, it can be seen that the speech synthesis model 10 provided in this embodiment may include: an encoding network 11, a decoding network 12, and an attention network 13. The attention network 13 is positioned between the encoding network 11 and the decoding network 12.

[0087] The encoding network 11 and the decoding network 12 each include a recurrent neural network.

[0088] The encoding network 11 may include: an embedding layer 11a, a convolutional layer 11b, and a first recurrent neural network layer 11c.

[0089] The encoding network 11 is mainly used to receive the text to be synthesized and to use the embedding layer 11a to convert or map each element of the text to be synthesized into a mathematical vector expression; and to input the mathematical vector expression corresponding to each element of the text to be synthesized into the convolutional layer 11b for convolution processing to obtain the feature vector after convolution processing; and to output the feature vector obtained by convolution processing to the first recurrent neural network layer 11c. The first recurrent neural network layer 11c performs feature extraction, dimensionality increase and other processing on the feature vector output by the convolutional layer 11b to obtain the high-dimensional acoustic feature information corresponding to each element. The acoustic feature information corresponding to these elements is concatenated together according to the order of the elements to obtain the acoustic feature sequence corresponding to the text to be synthesized.

[0090] Among them, the embedding layer 11a can be as follows: Figure 3The embedding layer 11a is shown as part of the encoding network 11; in actual applications, the embedding layer 11a can also be a network layer independent of the encoding network 11, arranged before the encoding network 11, and whether the embedding layer 11a is bound with the encoding network 11 can be flexibly deployed according to requirements.

[0091] The decoding network 12 can include a second recurrent neural network layer 12a, a first fully connected layer 12b, a second fully connected layer 12c, and a linear layer 12d (pre-net layer).

[0092] The second recurrent neural network layer 12a is mainly used to receive a target vector corresponding to the current step from the attention network 13, and convert the target vector corresponding to the current step to obtain a target state quantity of the current step. The target vector corresponding to the current step is obtained by weighting calculation according to the attention vector output by the attention network 13 and the acoustic feature sequence corresponding to the text to be synthesized.

[0093] The second recurrent neural network layer 12a can obtain the target state quantity of the current step by the following way, which can include the following steps:

[0094] Step (a) converts the input target vector of the current step to obtain an initial state quantity corresponding to the current step.

[0095] Step (b) generates a mask according to the state transition control factor.

[0096] Step (c) performs weighting processing on the initial state quantity of the current step and the target state quantity of the previous step based on the mask to obtain the target state quantity of the current step.

[0097] Taking the second recurrent neural network layer 12a including an LSTM network as an example, the target state quantity corresponding to each step includes a first target state quantity and a second target state quantity, wherein the first target state quantity can be represented as hidden state, and the second target state quantity can be represented as cell state; accordingly, the target state quantity of the current step can be represented by formula (1) as follows:

[0098]

[0099] In formula (1), c t represents the first target state quantity of the current step, h t represents the second target state quantity of the current step, c t-1 represents the first target state quantity of the previous step, h t-1 represents the second target state quantity of the previous step, c t ′ represents the first initial state quantity of the current step, h t ′ represents the second initial state quantity of the current step, a mask generated according to the state transition control factor corresponding to the current step for c t a mask generated according to the state transition control factor corresponding to the current step for c a mask generated according to the state transition control factor corresponding to the current step for h t a mask generated according to the state transition control factor corresponding to the current step for h

[0100] It should be noted that the target state quantity output by the second recurrent neural network layer 12a at each step has a corresponding relationship with the target spectrum of the step output by the final output, wherein the target spectrum output by each step is the observation quantity, and the target state quantity of the step is the intermediate hidden quantity of the decoding network 12 performing spectrum feature extraction.

[0101] The target state quantity of the current step is output to the first full connection layer 12b and the second full connection layer 12c respectively.

[0102] When the second recurrent neural network layer 12a is implemented by LSTM, the first target state quantity of the current step can be output to the first full connection layer 12b and the second full connection layer 12c respectively; and the second target state quantity of the current step is recorded by the second recurrent neural network layer 12a for calculating the initial state quantity of the next step.

[0103] The first full connection layer 12b is mainly used to convert the received target state corresponding to the current step into the target spectrum corresponding to the current step. The implementation of the first full connection layer 12b is not limited in the present disclosure.

[0104] The second full connection layer 12c is used to convert the received target state quantity corresponding to the current step into a one-dimensional stop token. The present disclosure does not limit the calculation method of the second full connection layer 12c for obtaining the stop token, for example, the second full connection layer 12c can obtain a one-dimensional stop token by performing weighted calculation on the target state quantity of the current step; for another example, the second full connection layer 12c can obtain a weighted calculation result by performing weighted calculation on the target state quantity of the current step, and then use the sigmoid function to classify the weighted calculation result to obtain a one-dimensional stop token.

[0105] The stop token is used to indicate whether the current step reaches the end position of the text to be synthesized. If the stop token indicates that the current step has reached the end position of the text to be synthesized, the prediction of the target spectrum for the text to be synthesized is ended; if the stop token indicates that the current step has not reached the end position of the text to be synthesized, the prediction of the target spectrum for the text to be synthesized needs to be continued.

[0106] If the stop token indicates that the current step has not reached the end position of the text to be synthesized, the linear layer 12d extracts effective information from the existing target spectrum and passes the extracted effective information to the attention network 13, so that the attention network 13 updates the attention vector and predicts the target spectrum of the next step.

[0107] In a possible implementation, the linear layer 12d can directly input the effective information extracted from the existing target spectrum to the attention network 13. As shown by the dotted line with an arrow in FIG. 4. Figure 3

[0108] In another possible implementation, the linear layer 12d can also input the effective information extracted from the existing target spectrum to the second recurrent neural network layer 12a, so as to pass the effective information extracted from the existing target spectrum to the attention network 13 through the second recurrent neural network layer 12a. As shown by the solid line with an arrow in FIG. 4, the second recurrent neural network layer 12a points to the attention network 13. Figure 3

[0109] The second recurrent neural network layer 12a can convert the effective information extracted from the existing target spectrum information by using a preset algorithm, and the preset algorithm is not limited in the present disclosure. Of course, the second recurrent neural network layer 12a can also not process the effective information extracted from the existing target spectrum information.

[0110] The linear layer 12d can be understood as a preposed auxiliary network layer, which ignores the invalid information in the target spectrum by extracting the effective information in the existing target spectrum. For example, the linear layer 12d can extract the effective information of the target spectrum at the current step, to predict the target spectrum at the next step; or the linear layer 12d can also extract the effective information of the target spectrum corresponding to the inverse multiple steps, to predict the target spectrum at the next step, and the present disclosure is not limited in this regard.

[0111] The attention network 13 is mainly used to receive the effective information of the existing target spectrum (for example, the effective information of the target spectrum at the current step) output by the second recurrent neural network layer 12a, and generate an updated attention vector according to the received effective information of the target spectrum. The updated attention vector is used for weighted calculation with the acoustic feature sequence corresponding to the text to be synthesized, and the result of the weighted calculation is used as the input of the decoding network 12 to predict the target spectrum at the next step.

[0112] In this scheme, the information of the acoustic feature sequence and the attention mechanism are both transmitted to the decoding network 12 by performing weighted calculation on the attention vector and the acoustic feature sequence, and transmitting the result of the weighted calculation to the decoding network 12. Therefore, the decoding network 12 can determine which relevant regions of the acoustic feature sequence should be focused on when predicting the target spectrum, so as to ignore the irrelevant features or the regions with low relevance included in the acoustic feature sequence. In addition, the result of the weighted calculation has a low dimension, which is conducive to reducing the calculation amount of the decoding network 12.

[0113] ​​In addition, when the decoding network 12 predicts the target spectrum of the first step, the initial value of the attention vector can be preset, or can also be determined by the electronic device by analyzing the acoustic feature sequence corresponding to the text to be synthesized, and the present disclosure does not limit the implementation manner of determining the initial value of the attention vector.

[0114] It should also be noted that when the decoding network 12 masks and weights the initial state quantity of the current step and the target state quantity of the previous step to generate the target spectrum of the current step when predicting the target spectrum, the greater the role of the target state quantity of the previous step, the closer the target state quantity of the current step to the target state quantity of the previous step, and the closer the target spectrum of the current step to the target spectrum of the previous step; the smaller the role of the target spectrum of the previous step, the greater the difference between the target state quantity of the current step and the target state quantity of the previous step, and the greater the difference between the target spectrum of the current step and the target spectrum of the previous step. The role of the target state quantity of the previous step in the mask weighting fusion is realized through the state transition control factor.

[0115] When the role of the target state quantity of the previous step is greater, the target spectrum of the current step is closer to the target spectrum of the previous step, the attention network 13 extracts the effective information of the target spectrum of the current step based on the linear layer 12d as the query quantity of the attention, the acoustic feature sequence used to predict the target spectrum of the next step is closer to the acoustic feature sequence used to predict the target spectrum of the current step, and the attention vector generated by the attention network is closer to the attention vector of the previous step. This is manifested as a slower moving speed on the text position of the text to be synthesized. The slower the moving speed on the text position, the more the number of target spectrums obtained, and the slower the speech rate of the target audio obtained; similarly, when the role of the target state quantity of the previous step is smaller, the target spectrum of the current step is greater than the target spectrum of the previous step, the attention network 13 extracts the effective information of the target spectrum of the current step based on the linear layer 12d as the query quantity of the attention, the acoustic feature sequence used to predict the target spectrum of the next step is greater than the acoustic feature sequence used to predict the target spectrum of the current step, and the attention vector generated by the attention network is greater than the attention vector of the previous step. This is manifested as a faster moving speed on the text position of the text to be synthesized. The faster the moving speed on the text position, the fewer the number of target spectrums obtained, and the faster the speech rate of the target audio obtained. Through the above manner, the number of target spectrums is controlled through the state transition control factor, that is, the number of target spectrums is controlled through the state transition control factor, and the speech rate of the target audio is controlled.

[0116] As a possible implementation, the first recurrent neural network layer 11c can include any type of recurrent neural network, such as a long short term memory (LSTM), a gate recurrent unit (GRU), a simple recurrent unit (SRU), etc.

[0117] As a possible implementation, the second recurrent neural network layer 12a can include any type of recurrent neural network, such as an LSTM, a GRU, an SRU, etc. As a possible implementation, the second recurrent neural network layer 12a can include multiple recurrent neural networks, for example Figure 3 In the illustrated embodiment, the second recurrent neural network layer 12a includes two LSTM networks connected in sequence.

[0118] In addition, the first recurrent neural network layer 11c and the second recurrent neural network layer 12a can also include other types of recurrent neural networks. The above are only examples and are not a limitation on the types of networks used by the first recurrent neural network layer 11c and the second recurrent neural network layer 12a.

[0119] In addition, the first recurrent neural network layer 11c and the second recurrent neural network layer 12a can employ the same type of recurrent neural network or different types of recurrent neural networks, which are not limited by the present disclosure.

[0120] In Figures 1 to 3 Based on the illustrated embodiment, the speech speed adjustment method provided by the present disclosure will be described in detail below through a specific example.

[0121] Suppose the text to be synthesized is A, which includes N phonemes, where the first phoneme is denoted as A1, the second phoneme is denoted as A2, the third phoneme is denoted as A3, and so on, and the last phoneme is denoted as AN.

[0122] In combination with Figure 2 In combination with Figure 3 The speech synthesis model 10 provided by the illustrated embodiment inputs the text to be synthesized A into the encoding network 11, and the encoding network 11 outputs the acoustic feature sequence X corresponding to the text to be synthesized A. The implementation of the encoding network 11 converting the text to be synthesized A into the acoustic feature sequence X can refer to the description of the foregoing embodiments, and for the sake of brevity, will not be described here.

[0123] In predicting the target spectrum of the first step, the initial value of the attention vector provided by the attention network 13 is denoted as S0, S0 is weighted with the acoustic feature sequence X to obtain the target vector Y1 corresponding to the first step, and the target vector Y11 is input into the decoding network 12.

[0124] The second recurrent neural network layer 12a of the decoding network 12 obtains the initial state quantity corresponding to the first step by converting the target vector Y11, and outputs the target state quantity corresponding to the first step by performing mask weighting fusion on the initial state quantity corresponding to the first step and the target state quantity of the previous step through the state transition control factor K1 corresponding to the first step. It should be noted that the target state quantity of the previous step used in the mask weighting fusion of the first step can be preset, for example, the target state quantity of the previous step can be 0.

[0125] When the second recurrent neural network layer 12a includes two layers of connected LSTMs, the initial state quantity corresponding to the first step includes a first initial state quantity and a second initial state quantity, and the target state quantity of the first step includes a first target state quantity and a second target state quantity. The first initial state quantity and the second initial state quantity are weighted and fused according to the mask to obtain the first target state quantity and the second target state quantity, which can be calculated in combination with the aforementioned formula (1). For the sake of brevity, details are not repeated here.

[0126] It is assumed that the target state quantity corresponding to the first step includes a first target state quantity c1 and a second target state quantity h1.

[0127] The second target state quantity h1 of the first step is input into the first fully connected layer 12b and the second fully connected layer 12c, respectively.

[0128] The first fully connected layer 12b outputs the target spectrum P1 of the first step by converting the second target state quantity h1.

[0129] The second fully connected layer 12c outputs the stop symbol R1 corresponding to the first step by calculating the second target state quantity h1.

[0130] For example, if the second fully connected layer 12c obtains a weighted calculation result by performing weighted calculation on the target vector Y1, the weighted calculation result is the stop symbol R1. If the stop symbol R1 is greater than or equal to a preset threshold, it is determined that the stop symbol indicates that the end position of the text to be synthesized is reached. If the stop symbol R1 is less than the preset threshold, it is determined that the stop symbol indicates that the end position of the text to be synthesized is not reached.

[0131] For example, if the second fully connected layer 12c obtains a weighted calculation result by performing weighted calculation on the target vector Y1, and the weighted calculation result is classified using a sigmoid function to obtain a classification result, the classification result is the stop symbol R1. Assuming that the classification result is represented by 0 or 1, if the stop symbol R1 is 1, it is determined that the stop symbol indicates that the end position of the text to be synthesized is reached. If the stop symbol R1 is 0, it is determined that the stop symbol indicates that the end position of the text to be synthesized is not reached.

[0132] If the stop symbol R1 indicates that the end position of the text A to be synthesized has not been reached, the linear layer 12d extracts the effective information of the target spectrum P1 and delivers the effective information of the target spectrum P1 to the attention network 13.

[0133] In a possible implementation, the linear layer 12d can directly input the effective information of the target spectrum P1 to the attention network 13. In another possible implementation, the linear layer 12d can also input the effective information of the target spectrum P1 to the second recurrent neural network layer 12a, so as to deliver the effective information of the target spectrum P1 to the attention network 13 through the second recurrent neural network layer 12a.

[0134] The second recurrent neural network layer 12a can convert the effective information of the target spectrum information P1 by using a preset algorithm, and the preset algorithm is not limited in the present disclosure. Of course, the second recurrent neural network layer 12a can also not process the effective information of the target spectrum information P1.

[0135] The attention network 13 receives the effective information of the target spectrum P1 as a query, queries the acoustic feature sequence X, and outputs an updated attention vector S1. The attention vector S1 is weighted and calculated with the acoustic feature sequence X to obtain a target vector Y2 corresponding to the second step, and the target vector Y2 is input to the decoding network 12, so that the decoding network 12 predicts the target spectrum corresponding to the second step.

[0136] The implementation manner of the decoding network 12 predicting the target spectrum corresponding to the second step is similar to the implementation manner of the decoding network 12 predicting the target spectrum corresponding to the first step, and will not be described here.

[0137] Suppose that the first full connection layer 12b outputs the target spectrum P2 of the second step, and the second full connection layer 12c outputs the stop symbol R2 corresponding to the second step. If the stop symbol R2 indicates that the end position of the text A to be synthesized has not been reached, the linear layer 12d extracts the effective information of the target spectrum P2 and delivers the effective information of the target spectrum P2 to the attention network 13, so that the attention network 13 queries the acoustic feature sequence X according to the effective information of the target spectrum P2 and outputs an attention vector S2 used for predicting the target spectrum of the third step. The attention vector S2 is weighted and calculated with the acoustic feature sequence X to obtain a target vector Y3 corresponding to the third step, and the target vector Y3 is input to the decoding network 12, so that the decoding network 12 predicts the target spectrum P3 corresponding to the third step.

[0138] In this way, until the stop symbol indicates that the end position of the text A to be synthesized is reached, the prediction of the target spectrum for the text A to be synthesized is stopped.

[0139] In the above process, the state transition control factor corresponding to each step can be dynamically changed, so as to flexibly control the speech rate of different audio segments in the final synthesized target audio in the speech synthesis process. The implementation of updating the state transition control factor corresponding to each step can refer to the detailed description in the foregoing, and will not be described here for the sake of simplicity.

[0140] Then, the target spectrum corresponding to each step output by the decoding network 12 is spliced in the order, that is, the target spectrum corresponding to the text A to be synthesized. By playing the target spectrum at a preset rate, the target audio with the target speech rate can be obtained.

[0141] The method of the embodiment, by obtaining the text to be synthesized, inputs the text to be synthesized into a speech synthesis model, the speech synthesis model comprising an encoding network, an attention network and a decoding network, wherein the encoding network converts the input text to be synthesized into an acoustic feature sequence; the attention network is used to output an attention vector, and the decoding network is used to output a target spectrum according to the attention vector, the acoustic feature sequence and the state transition control factor; and then the target audio with the target speech rate is obtained through the target spectrum. The present disclosure introduces the state transition control factor into the speech synthesis model, dynamically controls the number of target spectrums by using the state transition control factor, so as to flexibly adjust the speech rate in the speech synthesis process. And the audio synthesized by the method provided by the present disclosure has high natural sound degree, which is beneficial to improve the user experience.

[0142] Exemplarily, the present disclosure also provides a speech rate adjustment device.

[0143] Figure 4 The structure diagram of the speech rate adjustment device provided by an embodiment of the present disclosure is shown in FIG. 4. As shown in FIG. 4, the speech rate adjustment device 400 provided by the embodiment can comprise: Figure 4

[0144] The acquisition module 401 is configured to acquire the text to be synthesized.

[0145] The spectrum feature extraction module 402 is configured to input the text to be synthesized into a speech synthesis model, and acquire the target spectrum corresponding to the text to be synthesized output by the speech synthesis model; wherein the speech synthesis model comprises an encoding network, an attention network and a decoding network, the encoding network is configured to convert the input text to be synthesized into an acoustic feature sequence; the attention network is configured to output an attention vector, and the decoding network is configured to output the target spectrum corresponding to the text to be synthesized according to the input attention vector, acoustic feature sequence and state transition control factor; and the state transition control factor is configured to control the number of target spectrums.

[0146] ​The audio processing module 403 is configured to acquire target audio according to the target spectrum corresponding to the text to be synthesized, and the target audio has a target speech speed.

[0147] As a possible implementation, if the state transition control factor is less than a preset threshold, the target speech speed of the target audio is less than a reference speech speed; if the state transition control factor is greater than the preset threshold, the target speech speed of the target audio is greater than the reference speech speed; and if the state transition control factor is equal to the preset threshold, the target speech speed of the target audio is equal to the reference speech speed.

[0148] As a possible implementation, the decoding network comprises a first full connection layer, a second full connection layer, a linear layer and a recurrent neural network layer.

[0149] Correspondingly, the spectrum feature extraction module 402 is specifically configured to perform weighted calculation on the attention vector and the acoustic feature sequence to obtain a target vector of a current step, and input the target vector of the current step into the recurrent neural network layer; the recurrent neural network layer acquires a target state quantity of the current step according to the target vector of the current step, a state transition control factor and a target state quantity of a previous step; the target state quantity of the current step is input into the first full connection layer to acquire the target spectrum of the current step output by the first full connection layer; the target state quantity of the current step is input into the second full connection layer to acquire a stop symbol output by the second full connection layer; if the stop symbol indicates that an end position of the text to be synthesized has not been reached, the target spectrum of the current step is extracted through the linear layer and input into the attention network, so that the attention network updates the attention vector; the weighted calculation on the attention vector and the acoustic feature sequence to obtain the target vector of the current step is performed again, and the target vector of the current step is input into the recurrent neural network layer; the recurrent neural network layer acquires the target state quantity of the current step according to the target vector of the current step, the state transition control factor and the target state quantity of the previous step; and the target state quantity of the current step is input into the first full connection layer to acquire the target spectrum of the current step output by the first full connection layer; the target state quantity of the current step is input into the second full connection layer to acquire the stop symbol output by the second full connection layer, until the stop symbol indicates that the end position of the text to be synthesized has been reached.

[0150] As a possible implementation, the spectrum feature extraction module 402 is specifically configured to perform mask weighted fusion according to the target vector of the current step, the state transition control factor and the target state quantity of the previous step to acquire the target state quantity of the current step.

[0151] As a possible implementation, the spectrum feature extraction module 402 is specifically configured to obtain an initial state quantity of a current step according to the target vector; generate a mask according to the state transition control factor; and perform weighted fusion on the initial state quantity of the current step and a target state quantity of a previous step according to the mask, to obtain the target state quantity of the current step.

[0152] As a possible implementation, the spectrum feature extraction module 402 is further configured to, before obtaining the target state quantity of the current step according to the target vector of the current step, the state transition control factor, and the target state quantity of the previous step, update a size of the state transition control factor corresponding to the current step.

[0153] As a possible implementation, the spectrum feature extraction module 402 is specifically configured to update the size of the state transition control factor corresponding to the current step according to one or more of the target speech rate, an acoustic feature sequence corresponding to the text to be synthesized, and an importance of text content corresponding to the current step.

[0154] The speech rate adjustment apparatus provided in this embodiment is used to implement the technical solutions provided in any of the method embodiments, and has similar implementation principles and technical effects. For brevity, the detailed description is referred to the foregoing method embodiments.

[0155] Exemplarily, the present disclosure further provides an electronic device.

[0156] Figure 5 A structural schematic diagram of an electronic device provided in an embodiment of the present disclosure is shown in FIG. 5. As shown in FIG. 5, the electronic device 500 provided in this embodiment includes a memory 501 and a processor 502. Figure 5

[0157] The memory 501 can be an independent physical unit and can be connected to the processor 502 through a bus 503. The memory 501 and the processor 502 can also be integrated together and implemented by hardware.

[0158] The memory 501 is used to store program instructions, and the processor 502 invokes the program instructions to execute the speech rate adjustment method provided in any of the method embodiments.

[0159] Optionally, when part or all of the method in the above embodiment is implemented by software, the electronic device 500 can only include the processor 502. The memory 501 for storing the program is located outside the electronic device 500, and the processor 502 is connected to the memory through a circuit / wire, for reading and executing the program stored in the memory.

[0160] ​The processor 502 can be a central processing unit (CPU), a network processor (NP), or a combination of CPU and NP.

[0161] The processor 502 can further include a hardware chip. The hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0162] The memory 501 can include a volatile memory, such as a random-access memory (RAM); the memory can also include a non-volatile memory, such as a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); the memory can also include a combination of the above-mentioned types of memories.

[0163] The present disclosure also provides a computer-readable storage medium (which can also be referred to as a readable storage medium), which includes computer program instructions, and the computer program instructions, when executed by at least one processor of an electronic device, cause the electronic device to implement the speech speed adjustment method provided by any one of the above method embodiments.

[0164] The present disclosure also provides a computer program product, which includes computer program instructions stored in a readable storage medium, and at least one processor of an electronic device can read the computer program instructions from the readable storage medium, and the at least one processor executes the computer program instructions to cause the electronic device to implement the speech speed adjustment method provided by any one of the above method embodiments.

[0165] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A speech rate adjustment method characterized by comprising: The method comprises: acquiring a text to be synthesized; inputting the text to be synthesized into a speech synthesis model to acquire a target spectrum corresponding to the text to be synthesized output by the speech synthesis model; wherein the speech synthesis model comprises an encoding network, an attention network, and a decoding network; the encoding network is configured to convert the input text to be synthesized into an acoustic feature sequence; the attention network is configured to output an attention vector; the decoding network is configured to output the target spectrum corresponding to the text to be synthesized according to the input attention vector, the acoustic feature sequence, and a state transition control factor; and the state transition control factor is configured to control the number of target spectrums corresponding to the text to be synthesized; acquiring a target audio according to the target spectrum corresponding to the text to be synthesized, wherein the target audio has a target speech rate; wherein, when predicting the target spectrum, the decoding network performs weighted calculation on the attention vector and the acoustic feature sequence to obtain a target vector of a current step; acquires an initial state quantity of the current step according to the target vector of the current step; generates a mask according to the state transition control factor, and performs weighted fusion on the initial state quantity of the current step and a target state quantity of a previous step according to the mask to acquire the target state quantity of the current step.

2. The method of claim 1, wherein, If the state transition control factor is less than a preset threshold, the target speech rate of the target audio is less than a reference speech rate. If the state transition control factor is greater than the preset threshold, the target speech rate of the target audio is greater than the reference speech rate. If the state transition control factor is equal to the preset threshold, the target speech rate of the target audio is equal to the reference speech rate.

3. The method of claim 1, wherein, The decoding network comprises a first full connection layer, a second full connection layer, a linear layer, and a recurrent neural network layer. The decoding network is configured to output the target spectrum corresponding to the text to be synthesized according to the input attention vector, the acoustic feature sequence, and the state transition control factor, comprising: performing weighted calculation on the attention vector and the acoustic feature sequence to obtain a target vector of a current step, and inputting the target vector of the current step into the recurrent neural network layer; the recurrent neural network layer acquires a target state quantity of the current step according to the target vector of the current step, a state transition control factor, and a target state quantity of a previous step; inputting the target state quantity of the current step into the first full connection layer to acquire the target spectrum of the current step output by the first full connection layer; and inputting the target state quantity of the current step into the second full connection layer to acquire a stop symbol output by the second full connection layer; if the stop symbol indicates that the end position of the text to be synthesized has not been reached, the target spectrum of the current step is extracted through the linear layer and input into the attention network, so that the attention network updates the attention vector. The method further comprises, before obtaining the target state quantity of the current step according to the target vector of the current step, the state transition control factor, and the target state quantity of the previous step:

4. The method of claim 3, wherein, updating the size of the state transition control factor corresponding to the current step. The updating the size of the state transition control factor corresponding to the current step comprises:

5. The method of claim 4, wherein, updating the size of the state transition control factor corresponding to the current step according to one or more of the target speech rate, the acoustic feature sequence corresponding to the text to be synthesized, and the importance of the text content corresponding to the current step. comprises:

6. A speech rate adjustment apparatus characterized by comprising: an obtaining module configured to obtain text to be synthesized; a spectrum feature extraction module configured to input the text to be synthesized into a speech synthesis model to obtain a target spectrum corresponding to the text to be synthesized output by the speech synthesis model; wherein the speech synthesis model comprises an encoding network, an attention network, and a decoding network, the encoding network is configured to convert the input text to be synthesized into an acoustic feature sequence; the attention network is configured to output an attention vector, and the decoding network is configured to output the target spectrum corresponding to the text to be synthesized according to the input attention vector, the acoustic feature sequence, and a state transition control factor; the state transition control factor is configured to control the number of target spectrums; wherein the decoding network, when predicting the target spectrum, performs weighted calculation on the attention vector and the acoustic feature sequence to obtain a target vector of a current step; obtains an initial state quantity of the current step according to the target vector of the current step; generates a mask according to the state transition control factor, and performs weighted fusion on the initial state quantity of the current step and the target state quantity of the previous step according to the mask to obtain the target state quantity of the current step; an audio processing module configured to obtain a target audio according to the target spectrum corresponding to the text to be synthesized, the target audio having a target speech rate. comprises:

7. An electronic device, comprising: a memory and a processor; the memory is configured to store computer program instructions; the processor is configured to execute the computer program instructions, so that the electronic device implements the speech rate adjustment method according to any one of claims 1 to 5. comprises:

8. A readable storage medium, characterized by, computer program instructions; the computer program instructions are executed by at least one processor of an electronic device, so that the electronic device implements the speech rate adjustment method according to any one of claims 1 to 5. ​

Images