Artificial intelligence-based speech generation method and apparatus, computer device, and medium

Abstract

The application belongs to the technical field of artificial intelligence, and relates to a voice generation method based on artificial intelligence, which is applied to the field of financial technology and comprises the following steps: obtaining an input text sequence; calling a text-to-speech model; wherein the text-to-speech model comprises a language model and an acoustic model; based on a target beam search algorithm, the language model is used to decode and process the text sequence to obtain a corresponding target candidate sequence set; a decoding result is determined from the target candidate sequence set; the decoding result is converted into corresponding voice data based on the acoustic model; and the voice data is output. The application also provides a voice generation device based on artificial intelligence, a computer device and a storage medium. In addition, the application also relates to blockchain technology, and voice data can be stored in a blockchain. The application effectively improves the stability and accuracy of generated voice data by adopting a decoding mode based on a target beam search algorithm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence development and the field of financial technology, and in particular relates to a voice generation method and device based on artificial intelligence, a computer device and a storage medium. BACKGROUND

[0002] In the financial field, with the continuous progress of technology, the Text-To-Speech (TTS) technology has made significant progress. This progress is largely due to the adoption of the Language Model (LM) based TTS paradigm. The LM based TTS technology realizes the natural conversion from text to speech by simulating the sequence of phonetic symbols, which is of great significance to improve the transmission efficiency of financial information and user experience.

[0003] In the LM based TTS model, the pre-trained neural phonetic marker plays a crucial role. These markers can quantify the sequence of phonetic symbols from continuous speech signals, providing accurate and rich speech data for the TTS model. By training with large speech data sets, the LM based TTS model can synthesize coherent and natural speech from a single text prompt, which not only preserves the information in the text, but also effectively conveys the sound and emotional characteristics of the original text.

[0004] However, although the LM based TTS model has made significant progress in synthesizing speech, there are still some technical challenges. Among them, the most prominent is the randomness problem introduced by the sampling based decoding strategy. In the decoding process, the introduction of randomness, although to some extent, increases the diversity of speech generation, but also can lead to the increase of instability of phonetic symbol generation. This instability often manifests as flaws and pronunciation errors in the speech, which in turn affects the overall quality of the generated speech, resulting in low stability and accuracy of speech generation. SUMMARY

[0005] The purpose of the embodiments of the present application is to propose a voice generation method, device, computer device and storage medium based on artificial intelligence, to solve the technical problem that the sampling based decoding strategy used in the existing LM based TTS model for synthesizing speech can lead to low stability and accuracy of speech generation.

[0006] In order to solve the above technical problems, the embodiments of the present application provide a voice generation method based on artificial intelligence, which adopts the following technical solutions:

[0007] obtaining an input text sequence;

[0008] invoke a preset text-to-speech model; wherein the text-to-speech model comprises a language model and an acoustic model;

[0009] based on a preset target beam search algorithm, decode the text sequence using the language model to obtain a corresponding target candidate sequence set;

[0010] determine a decoding result from the target candidate sequence set;

[0011] convert the decoding result into corresponding speech data based on the acoustic model;

[0012] output the speech data.

[0013] Further, the step of decoding the text sequence using the language model based on the preset target beam search algorithm to obtain a corresponding target candidate sequence set specifically comprises:

[0014] based on the target beam search algorithm, using the language model to perform candidate sequence expansion processing on the text sequence to obtain a corresponding first candidate sequence;

[0015] performing repetition detection and penalty processing on the first candidate sequence to obtain a second candidate sequence after score recalculation;

[0016] based on the score of the second candidate sequence, performing sequence set selection processing on the second candidate sequence that meets the preset screening condition to obtain a corresponding first candidate sequence set;

[0017] repeating the candidate sequence expansion processing, the repetition detection and penalty processing, and the sequence set selection processing at each time step until the preset termination condition is met and a corresponding second candidate sequence set is obtained;

[0018] the second candidate sequence set is taken as the target candidate sequence set.

[0019] Further, the step of determining a decoding result from the target candidate sequence set specifically comprises:

[0020] obtain the score of each candidate sequence contained in the target candidate sequence set;

[0021] numerically compare the scores of all candidate sequences to filter out a specified candidate sequence with the highest score from all candidate sequences;

[0022] the specified candidate sequence is taken as the decoding result.

[0023] Further, the step of outputting the speech data specifically comprises:

[0024] perform quality optimization processing on the voice data based on a preset quality optimization strategy to obtain corresponding target voice data;

[0025] obtain a preset specified output mode;

[0026] output the target voice data based on the specified output mode.

[0027] Further, the step of performing quality optimization processing on the voice data based on a preset quality optimization strategy to obtain corresponding target voice data specifically includes:

[0028] perform noise reduction processing on the voice data to obtain corresponding first voice data;

[0029] perform echo cancellation processing on the first voice data to obtain corresponding second voice data;

[0030] perform signal enhancement processing on the second voice data to obtain corresponding third voice data;

[0031] use the third voice data as the target voice data.

[0032] Further, before the step of decoding the text sequence using the language model based on the preset target beam search algorithm to obtain a corresponding target candidate sequence set, further includes:

[0033] obtain an original beam search algorithm;

[0034] obtain a preset time repetition awareness diversification strategy;

[0035] perform optimization processing on the beam search algorithm based on the time repetition awareness diversification strategy to obtain a corresponding optimization algorithm;

[0036] use the optimization algorithm as the target beam search algorithm.

[0037] Further, after the step of converting the decoding result into corresponding voice data based on the acoustic model, further includes:

[0038] invoke a preset specified storage medium;

[0039] determine a specified storage location corresponding to voice storage in the specified storage medium;

[0040] store the voice data into the specified storage location in the specified storage medium.

[0041] To solve the above technical problems, the embodiment of the application further provides a speech generation device based on artificial intelligence, which adopts the technical scheme as follows:

[0042] The first acquisition module is configured to acquire an input text sequence.

[0043] The first calling module is configured to call a preset text-to-speech model, wherein the text-to-speech model comprises a language model and an acoustic model.

[0044] The decoding module is configured to decode the text sequence based on a preset target beam search algorithm and using the language model to obtain a corresponding target candidate sequence set.

[0045] The first determination module is configured to determine a decoding result from the target candidate sequence set.

[0046] The conversion module is configured to convert the decoding result into corresponding speech data based on the acoustic model.

[0047] The output module is configured to output the speech data.

[0048] To solve the above technical problems, the embodiment of the application further provides a computer device, which adopts the technical scheme as follows:

[0049] Acquire an input text sequence.

[0050] Call a preset text-to-speech model, wherein the text-to-speech model comprises a language model and an acoustic model.

[0051] Decode the text sequence based on a preset target beam search algorithm and using the language model to obtain a corresponding target candidate sequence set.

[0052] Determine a decoding result from the target candidate sequence set.

[0053] Convert the decoding result into corresponding speech data based on the acoustic model.

[0054] Output the speech data.

[0055] To solve the above technical problems, the embodiment of the application further provides a computer readable storage medium, which adopts the technical scheme as follows:

[0056] Acquire an input text sequence.

[0057] Call a preset text-to-speech model, wherein the text-to-speech model comprises a language model and an acoustic model.

[0058] decoding processing on the text sequence using the language model based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set;

[0059] determining a decoding result from the target candidate sequence set;

[0060] converting the decoding result into corresponding voice data based on the acoustic model;

[0061] outputting the voice data.

[0062] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0063] The present application first acquires an input text sequence; then calls a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model; then decodes processing on the text sequence using the language model based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set; subsequently determines a decoding result from the target candidate sequence set; further converts the decoding result into corresponding voice data based on the acoustic model; finally outputs the voice data. The present application acquires an input text sequence, then decodes processing on the text sequence using a target beam search algorithm and a language model to obtain a target candidate sequence set, determines a decoding result from the target candidate sequence set, and further converts the decoding result into corresponding voice data based on the use of the acoustic model and outputs, thereby efficiently and accurately completing the voice generation processing of the text sequence. The present application adopts a decoding method based on a target beam search algorithm, effectively reduces the defects and pronunciation errors caused by the decoding method based on sampling in the prior art, and effectively improves the stability and accuracy of the generated voice data. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the schemes in the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0065] Figure 1 is an exemplary system architecture diagram to which the present application can be applied;

[0066] Figure 2 a flowchart of one embodiment of the voice generation method based on artificial intelligence according to the present application;

[0067] Figure 3 is a structural schematic diagram of one embodiment of the voice generation device based on artificial intelligence according to the present application;

[0068] Figure 4 is a structural schematic diagram of one embodiment of the computer device according to the present application. DETAILED DESCRIPTION

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application; the use herein of terms such as "comprise" and "have" and any variations thereof are intended to cover a non-exclusive inclusion; the use herein of terms such as "first", "second" and the like are intended to distinguish between similar objects unless the context indicates otherwise.

[0070] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase that in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that the embodiments described herein are merely examples from a multitude of embodiments that are in substantial compliance with the principles of the application.

[0071] In order to make the technical personnel in the art better understand the scheme of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings.

[0072] As shown in Figure 1 The system architecture 100 can include a terminal device 101, a network 102 and a server 103, the terminal device 101 can be a notebook computer 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103. The network 102 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0073] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0074] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing, in addition to the notebook computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an electronic book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer and a desktop computer, etc.

[0075] The server 103 can be a server providing various services, for example, a background server providing support for a page displayed on the terminal device 101.

[0076] It should be noted that the artificial intelligence-based speech generation method provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based speech generation apparatus is generally arranged in a server / terminal device.

[0077] It should be understood that Figure 1 The number of terminal devices, networks and servers in

[0078] With reference to Figure 2 , a flowchart of one embodiment of the artificial intelligence-based speech generation method according to the present application is shown. The order of the steps in the flowchart can be changed according to different needs, and some steps can be omitted. The artificial intelligence-based speech generation method provided in the embodiments of the present application can be applied to any scenario requiring speech generation, and then the artificial intelligence-based speech generation method can be applied to products in these scenarios, for example, speech generation in the field of finance and insurance. The artificial intelligence-based speech generation method includes the following steps:

[0079] Step S201, obtaining an input text sequence.

[0080] In the present embodiment, the electronic device (for example Figure 1The server / terminal device shown) can obtain the image to be checked for duplication through a wired connection or a wireless connection. It should be noted that the wireless connection can include, but is not limited to, 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other now known or future developed wireless connection methods. The execution subject of the present application is a voice generation system, or simply referred to as a system. The above text sequence can be a text input by a user that needs to be processed by text-to-speech. The text sequence can be provided by the user or automatically generated by the system through a certain method (such as a text generation model). For example, in the business scenario of financial insurance, the above text sequence can include transaction text, payment text, consultation text, and other text data.

[0081] In step S202, a preset text-to-speech model is called; wherein the text-to-speech model includes a language model and an acoustic model.

[0082] In the present embodiment, the above text-to-speech model can specifically adopt a LM-based TTS model, which includes a trained language model and an acoustic model. The language model is used to provide the probability distribution of the next symbol in the text sequence, and the acoustic model is used to convert the text sequence into speech.

[0083] In step S203, the language model is used to decode the text sequence based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set.

[0084] In the present embodiment, the specific implementation process of using the language model to decode the text sequence based on the preset target beam search algorithm to obtain the corresponding target candidate sequence set will be further described in detail in the subsequent specific embodiments, and will not be described here.

[0085] In step S204, a decoding result is determined from the target candidate sequence set.

[0086] In the present embodiment, the specific implementation process of determining the decoding result from the target candidate sequence set will be further described in detail in the subsequent specific embodiments, and will not be described here.

[0087] In step S205, the decoding result is converted into corresponding speech data based on the acoustic model.

[0088] In the embodiment, the decoding result is passed to the acoustic model. The acoustic model generates corresponding speech features (such as pitch, intensity, duration, etc.) according to the decoding result. And using speech synthesis technology (such as waveform synthesis, parameter synthesis, etc.), the speech features are converted into actual speech waveform, that is, the above-mentioned speech data is obtained.

[0089] In step S206, the speech data is outputted.

[0090] In the embodiment, the specific implementation process of outputting the speech data will be further described in detail in subsequent embodiments, and will not be described here.

[0091] Firstly, the input text sequence is obtained; then a preset text-to-speech model is called; wherein the text-to-speech model includes a language model and an acoustic model; then the language model is used to decode the text sequence based on a preset target beam search algorithm, to obtain a corresponding target candidate sequence set; subsequently, a decoding result is determined from the target candidate sequence set; further, the decoding result is converted into corresponding speech data based on the acoustic model; finally, the speech data is outputted. By obtaining the input text sequence, then decoding the text sequence based on the target beam search algorithm and the use of the language model to obtain the target candidate sequence set, and determining the decoding result from the target candidate sequence set, and then converting the decoding result into corresponding speech data based on the use of the acoustic model and outputting, the speech generation processing of the text sequence is efficiently and accurately completed. By using the decoding method based on the target beam search algorithm, the defects and pronunciation errors caused by the decoding method based on sampling in the prior art are effectively reduced, and the stability and accuracy of the generated speech data are effectively improved.

[0092] In some optional implementations, step S203 includes the following steps:

[0093] Based on the target beam search algorithm, the language model is used to perform candidate sequence expansion processing on the text sequence, to obtain corresponding first candidate sequences.

[0094] In the embodiment, the beam search algorithm is a heuristic search algorithm used to find the optimal or approximately optimal sequence under limited computing resources. It retains a fixed number of candidate sequences (i.e. beam width), and expands these candidate sequences at each step, and finally selects the sequence with the highest score as the result. The above-mentioned target beam search algorithm is an algorithm obtained by optimizing the beam search algorithm using a preset time repetition perception diversification strategy.

[0095] Specifically, a trained language model is loaded, which can predict the probability distribution of the next word or character for a given text sequence. The beam width B of the target beam search algorithm is also pre-set. The beam width B is an integer, representing the number of highest scoring candidate sequences to be retained during the beam search process. The selection of the beam width needs to balance the computing resources and the decoding quality. An empty candidate sequence set corresponding to the above text sequence is created. At time step t=0, a start symbol (e.g. <s>) as the starting point of the candidate sequence. The candidate sequence expansion process includes: sorting the sequences in the current candidate sequence set according to the scores. The top B sequences with the highest scores are selected from the sorted sequences. Then for each selected candidate sequence, all possible next tokens (words or characters) are obtained using the language model. These possible tokens are traversed, and a new expanded sequence is generated for each token. Then for each expanded sequence, its score is calculated. Specifically, the score of the current sequence can be added to the logarithmic probability of the next token.

[0096] The repetition detection and penalty process is performed on the first candidate sequence to obtain a second candidate sequence after the score is recalculated.

[0097] In this embodiment, the repetition detection and penalty process includes: detecting repeated tokens using a sliding window technique, and penalizing the detected repeated tokens. Specifically, detecting repeated tokens using a sliding window technique includes: defining the length of a sliding window, and for each expanded sequence, comparing the tokens in the current window with the tokens in the previous window to detect temporal repetition using the sliding window. The penalty for the detected repeated tokens includes: modifying the logarithmic probability of each detected repeated token according to a repetition penalty term. The repetition penalty term includes two hyperparameters (temporal repetition hyperparameter and inter-beam repetition hyperparameter) for adjusting the penalty of repeated tokens (controlling the penalty of temporal repetition and inter-beam repetition, respectively), and the hyperparameters can be optimized according to experimental data. In addition, for the detected repeated tokens, the algorithm will penalize their logarithmic probability to reduce their scores. Specifically, the penalty calculation method of modifying the logarithmic probability of the repeated token = original logarithmic probability of the repeated token - temporal repetition hyperparameter - inter-beam repetition hyperparameter can be used to perform the penalty processing of the logarithmic probability.

[0098] Based on the score of the second candidate sequence, the second candidate sequence set selection process is performed on the second candidate sequence to obtain a corresponding first candidate sequence set.

[0099] In this embodiment, the sequence set selection process includes: recalculating the score of each expanded sequence according to the modified logarithmic probability. Then the sequences after recalculating the score are sorted, and the top B sequences with the highest scores are selected from the sorted sequences as the new candidate sequence set.

[0100] The candidate sequence expansion process, the repetition detection and penalty process, and the sequence set selection process are repeatedly performed at each time step until the preset termination condition is met and the corresponding second candidate sequence set is obtained.

[0101] In this embodiment, the time step t is incremented and the steps of candidate sequence extension processing, repetition detection and penalty processing, and sequence set selection processing are repeatedly performed. This process continues until the end of the sequence is reached (e.g., a terminal symbol is encountered< / s> ) or other pre-set termination conditions (such as maximum length limit, decoding time limit, etc.) is added to the candidate sequence set, and the final second candidate sequence set, i.e., the target candidate sequence set, is obtained.

[0102] The second candidate sequence set is taken as the target candidate sequence set.

[0103] The present application uses the language model to perform candidate sequence expansion processing on the text sequence based on the target beam search algorithm, obtains the corresponding first candidate sequence, then performs repeated detection and penalty processing on the first candidate sequence, obtains the second candidate sequence after the score is recalculated, then performs sequence set selection processing on the second candidate sequence that meets the pre-set filtering conditions based on the score of the second candidate sequence, obtains the corresponding first candidate sequence set, and subsequently repeats the candidate sequence expansion processing, repeated detection and penalty processing, and sequence set selection processing at each time step until the pre-set termination condition is met and the corresponding second candidate sequence set is obtained. Finally, the second candidate sequence set is taken as the target candidate sequence set. The present application can effectively reduce the repetition in time and between beams by using the target beam search algorithm to cooperate with the language model to perform decoding processing on the text sequence. This decoding method can encourage the generation of more diverse outputs by penalizing repeated markers while maintaining decoding efficiency, thereby effectively improving the diversity and quality of the generated target candidate sequence set.

[0104] In some optional implementations of the present embodiment, step S204 includes the following steps:

[0105] The scores of each candidate sequence contained in the target candidate sequence set are obtained.

[0106] In the present embodiment, the target candidate sequence set contains multiple candidate sequences, and each candidate sequence corresponds to a corresponding score.

[0107] The scores of all the candidate sequences are compared numerically to filter out the specified candidate sequence with the highest score from all the candidate sequences.

[0108] In the embodiment, a corresponding comparison result is obtained by numerically comparing the scores of all candidate sequences, and then according to the comparison result, a specified score with the highest value can be screened from all scores, and a specified candidate sequence corresponding to the specified score can be screened from all candidate sequences.

[0109] The specified candidate sequence is taken as the decoding result.

[0110] The application obtains the scores of each candidate sequence included in the target candidate sequence set, then numerically compares the scores of all candidate sequences to screen a specified candidate sequence with the highest score from all candidate sequences, and subsequently takes the specified candidate sequence as the decoding result. The application obtains the scores of each candidate sequence included in the target candidate sequence set, then numerically compares the scores of all candidate sequences to screen a specified candidate sequence with the highest score from all candidate sequences and take it as the final decoding result, which effectively guarantees the accuracy of the obtained decoding result.

[0111] In some optional implementation manners, the step S206 includes the following steps:

[0112] The voice data is subjected to quality optimization processing based on a preset quality optimization strategy to obtain corresponding target voice data.

[0113] In the embodiment, the specific implementation process of the above-mentioned quality optimization processing of the voice data based on the preset quality optimization strategy to obtain corresponding target voice data will be further described in details in subsequent specific embodiments, and will not be elaborated here.

[0114] A preset specified output mode is obtained.

[0115] In the embodiment, the selection of the specified output mode is not specifically limited, and can be determined according to actual business requirements, for example, can include saving the generated voice data as an audio file, or directly playing the voice data on an audio playing device, etc.

[0116] The target voice data is output based on the specified output mode.

[0117] In the embodiment, the output processing of the target voice data can be completed by using the determined specified output mode.

[0118] The application obtains corresponding target voice data by performing quality optimization processing on the voice data based on a preset quality optimization strategy, then acquires a preset specified output mode, and subsequently outputs the target voice data based on the specified output mode. After the decoding result is converted into corresponding voice data based on an acoustic model, the application automatically and intelligently performs quality optimization processing on the voice data based on the use of a quality optimization strategy to obtain corresponding target voice data, effectively improving the quality of the generated target voice data. Subsequently, the target voice data is output based on a specified output mode, improving the output intelligence of the target voice data and facilitating improvement of the user's use experience.

[0119] In some optional implementations, the quality optimization processing on the voice data based on a preset quality optimization strategy to obtain corresponding target voice data includes the following steps:

[0120] Performing noise reduction processing on the voice data to obtain corresponding first voice data.

[0121] In this embodiment, the noise reduction processing on the voice data can be performed by using spectral subtraction, specifically by analyzing the spectrum of the voice data, estimating the noise spectrum, and subtracting the noise component from the mixed signal to obtain clearer first voice data.

[0122] Performing echo cancellation processing on the first voice data to obtain corresponding second voice data.

[0123] In this embodiment, the echo cancellation processing includes distinguishing direct sound signals and signals reflected by the environment by using an adaptive filter such as a least mean square error (LMS) filter or a normalized least mean square error (NLMS) filter, thereby minimizing echo.

[0124] Performing signal enhancement processing on the second voice data to obtain corresponding third voice data.

[0125] In this embodiment, the signal enhancement processing includes selecting a suitable wavelet basis function and threshold to remove noise components and retain useful parts of the signal by using a multi-scale signal analysis method.

[0126] Taking the third voice data as the target voice data.

[0127] The application obtains corresponding first voice data by performing noise reduction processing on the voice data, then obtains corresponding second voice data by performing echo cancellation processing on the first voice data, then obtains corresponding third voice data by performing signal enhancement processing on the second voice data, and subsequently takes the third voice data as the target voice data. The application performs noise reduction processing, echo cancellation processing and signal enhancement processing on voice data based on the use of a quality optimization strategy, thereby efficiently and accurately completing quality optimization processing on voice data and effectively improving the voice quality of the generated target voice data.

[0128] In some optional implementations of the embodiment, before step S203, the electronic device can further perform the following steps:

[0129] An original beam search algorithm is obtained.

[0130] In the embodiment, the original beam search algorithm refers to a traditional beam search algorithm. The beam search algorithm is a heuristic search algorithm used to find the optimal or approximately optimal sequence under limited computing resources. It works by retaining a fixed number of candidate sequences (i.e., beam width) and expanding these candidate sequences at each step, finally selecting the highest-scoring sequence as the result.

[0131] Although the beam search algorithm is designed to identify the highest-probability sequence in autoregressive decoding, it faces some challenges when applied to language model-based text-to-speech conversion. These challenges mainly manifest in two aspects. First, the beam search algorithm can encounter a time series collapse problem, where the generated sequence contains repeated symbols or very short symbol combinations. This causes the generated speech to fall into silence or persistently exhibit artificial artifacts. The initial part shows an identifiable speech pattern corresponding to a speech prompt, while the subsequent generated part consists of noise or silence. This problem can not only occur in greedy decoding, but also when the temperature setting in the sampling strategy is too low, which reduces randomness and leads to minimal variability in sampling. In addition, the beam search algorithm also often faces the challenge of inter-beam diversity loss, which means that there is little difference between the token sequences generated by different beams. This leads to almost identical speech content generated by different beams.

[0132] A preset time repetition perception diversification strategy is obtained.

[0133] In this embodiment, the time repetition awareness diversification strategy is composed of time repetition awareness, repetition penalty, and diversification selection. Specifically, time repetition awareness includes detecting whether the current candidate sequence contains previously appeared tokens in each decoding process. This can be achieved through sliding window technology, i.e., using a fixed length window to compare the tokens in the current window with the tokens in the previous window. Repetition penalty includes penalizing the log probability of detected repeated tokens to reduce their scores. The size of the penalty term can be controlled by hyperparameters, which can be optimized according to experimental data. Diversification selection includes introducing repetition penalty to encourage the algorithm to explore more diverse paths during decoding, thereby increasing the diversity of decoding results. This helps to avoid getting stuck in local optimal solutions and generates more natural and fluent speech or text.

[0134] The beam search algorithm is optimized based on the time repetition awareness diversification strategy to obtain a corresponding optimized algorithm.

[0135] In this embodiment, in speech generation or natural language processing tasks, although the traditional beam search algorithm can find sequences with higher scores, it sometimes gets stuck in local optimal solutions and generates repetitive or redundant content. To solve this problem, the time repetition awareness diversification strategy is introduced into the traditional beam search algorithm to obtain the corresponding optimized algorithm as the final target beam search algorithm.

[0136] The optimized algorithm is used as the target beam search algorithm.

[0137] In this embodiment, the target beam search algorithm obtained by introducing a repetition penalty mechanism can avoid generating repetitive or redundant content, thereby improving the quality of decoding results. The algorithm also encourages exploring more diverse paths during decoding, thereby increasing the diversity of decoding results. In addition, the hyperparameters of the algorithm can be optimized according to experimental data to adapt to different application scenarios and requirements.

[0138] In addition, the processing procedure of the target beam search algorithm includes: at each time step t (starting from 0), the first B highest-score sequences are taken out from the current candidate sequence set. For each candidate sequence, the next possible symbol (word, character or phoneme) is predicted using the language model, and the sequence score after adding the symbol is calculated. Here, the score is usually the cumulative value of the logarithmic probability, which measures the rationality of the sequence. A mechanism of time repetition-aware diversification is introduced to detect whether there are repeated symbols in time in the newly generated sequence, and to punish these repeated symbols to reduce their scores. This helps to reduce the possibility of generating redundant sequences and improve the diversity of the output. According to the modified score, all the expanded sequences are reordered, and the first B highest-score sequences are selected as the new candidate set for the expansion of the next time step. The above steps are repeated until the preset sequence length, decoding time limit or other termination conditions are reached.

[0139] In addition, the target beam search algorithm can also be further optimized. For example, a dynamic beam width adjustment mechanism is introduced, and the joint probability of the language model and the acoustic model is considered during decoding. Reinforcement learning techniques are used to further optimize the decoding strategy, and a policy network is learned to guide the beam search process, thereby improving decoding efficiency while ensuring generation quality. Through the above improvements, the target beam search algorithm shows better performance than the traditional beam search in multiple benchmark tests, especially when dealing with long sequences and complex language structures.

[0140] The present application obtains an original beam search algorithm, then obtains a preset time repetition-aware diversification strategy, and then optimizes the beam search algorithm based on the time repetition-aware diversification strategy to obtain a corresponding optimized algorithm. The present application can quickly and accurately construct a corresponding target beam search algorithm by introducing a time repetition-aware diversification strategy to optimize the beam search algorithm, and the constructed target beam search algorithm can effectively improve decoding quality and increase decoding diversity. This is conducive to subsequent decoding processing of text sequences using the target beam search algorithm and the language model, which can effectively improve the diversity and quality of the generated target candidate sequence set.

[0141] In some optional implementations of the embodiment, after step S205, the electronic device can further perform the following steps:

[0142] The preset specified storage medium is called.

[0143] In the embodiment, the selection of the specified storage medium is not specifically limited, for example, any one of a local database, a disk, a network disk, a blockchain, etc. can be used.

[0144] Determine a designated storage location corresponding to voice storage in the designated storage medium.

[0145] In the embodiment, the setting of the designated storage location is not specifically limited, and can be set according to actual business requirements. For example, the designated storage location can be a pre-constructed storage location specially used for storing voice data generated in a text-to-voice business processing process.

[0146] Store the voice data into the designated storage location in the designated storage medium.

[0147] In the embodiment, the voice data can be stored into the designated storage location in the designated storage medium by determining the location information (such as path information or coordinate information) of the designated storage location in the designated storage medium, and then storing the voice data into the designated storage location in the designated storage medium according to the location information.

[0148] The application calls a preset designated storage medium, then determines a designated storage location corresponding to voice storage in the designated storage medium, and then stores the voice data into the designated storage location in the designated storage medium. After the decoding result is converted into corresponding voice data based on the acoustic model, the application intelligently determines the designated storage location corresponding to voice storage in the called designated storage medium, and then stores the voice data into the designated storage location in the designated storage medium, so as to effectively ensure the data security of the voice data and effectively improve the storage intelligence and storage standardization of the voice data.

[0149] In some optional implementation manners, the obtained user information seeks user consent and meets the requirements of relevant laws and relevant policies.

[0150] In addition, the non-company software tools or components appearing in the embodiments of the application are only examples and do not represent actual use.

[0151] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0152] It should be emphasized that, in order to further ensure the privacy and security of the voice data, the voice data can also be stored in a node of a block chain.

[0153] The blockchain referred to in the present application is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. Blockchain, in essence, is a decentralized database, which is a series of data blocks associated using cryptographic methods, each data block containing information of a batch of network transactions, used to verify the validity of the information (anti-fake) and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0154] Embodiments of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0155] Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. several major directions.

[0156] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by computer readable instructions instructing related hardware, and the computer readable instructions can be stored in a computer readable storage medium. The program can include the processes of the above-mentioned embodiments when executed, wherein the storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0157] It should be understood that although each step in the flowchart of the accompanying drawings is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other orders. Moreover, at least part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or sub-steps or stages of other steps.

[0158] Further referenceFigure 3 As an implementation of the method shown in the above Figure 2 , the present application provides an embodiment of a voice generation device based on artificial intelligence. The device embodiment corresponds to the method embodiment shown in the above Figure 2 . The device can be applied in various electronic devices.

[0159] As shown in the above Figure 3 , the voice generation device based on artificial intelligence 300 comprises a first obtaining module 301, a first calling module 302, a decoding module 303, a first determining module 304, a conversion module 305, and an output module 306. Wherein:

[0160] The first obtaining module 301 is configured to obtain an input text sequence.

[0161] The first calling module 302 is configured to call a preset text-to-speech model. The text-to-speech model comprises a language model and an acoustic model.

[0162] The decoding module 303 is configured to decode the text sequence based on a preset target beam search algorithm using the language model to obtain a corresponding target candidate sequence set.

[0163] The first determining module 304 is configured to determine a decoding result from the target candidate sequence set.

[0164] The conversion module 305 is configured to convert the decoding result into corresponding speech data based on the acoustic model.

[0165] The output module 306 is configured to output the speech data.

[0166] In the present embodiment, the above modules or units are respectively used to perform operations corresponding to the steps of the voice generation method based on artificial intelligence of the foregoing embodiments, which will not be described here.

[0167] In some optional implementations of the present embodiment, the decoding module 303 comprises:

[0168] A first processing sub-module is configured to perform candidate sequence expansion processing on the text sequence based on the target beam search algorithm using the language model to obtain a corresponding first candidate sequence.

[0169] A second processing sub-module is configured to perform repetition detection and penalty processing on the first candidate sequence to obtain a second candidate sequence after score recalculation.

[0170] The third processing sub-module is configured to perform sequence set selection processing on the second candidate sequence in accordance with a preset screening condition based on the score of the second candidate sequence, to obtain a corresponding first candidate sequence set.

[0171] The fourth processing sub-module is configured to repeatedly perform candidate sequence expansion processing, repeated detection and penalty processing, and sequence set selection processing at each time step until a preset termination condition is met and a corresponding second candidate sequence set is obtained.

[0172] The first determining sub-module is configured to take the second candidate sequence set as the target candidate sequence set.

[0173] In the embodiment, the operations performed by the above modules or units correspond one by one to the steps of the artificial intelligence-based speech generation method of the foregoing embodiments, and thus will not be described here again.

[0174] In some optional implementations of the embodiment, the first determining module 304 includes:

[0175] The first obtaining sub-module is configured to obtain the scores of the candidate sequences included in the target candidate sequence set.

[0176] The screening sub-module is configured to perform numerical comparison on the scores of all the candidate sequences, to screen a specified candidate sequence with the highest score from all the candidate sequences.

[0177] The second determining sub-module is configured to take the specified candidate sequence as the decoding result.

[0178] In the embodiment, the operations performed by the above modules or units correspond one by one to the steps of the artificial intelligence-based speech generation method of the foregoing embodiments, and thus will not be described here again.

[0179] In some optional implementations of the embodiment, the output module 306 includes:

[0180] The optimization sub-module is configured to perform quality optimization processing on the speech data based on a preset quality optimization strategy, to obtain corresponding target speech data.

[0181] The second obtaining sub-module is configured to obtain a preset specified output mode.

[0182] The output sub-module is configured to output the target speech data based on the specified output mode.

[0183] In the embodiment, the operations performed by the above modules or units correspond one by one to the steps of the artificial intelligence-based speech generation method of the foregoing embodiments, and thus will not be described here again.

[0184] In some optional implementation manners of the embodiment, the optimization submodule comprises:

[0185] The first processing unit is configured to perform noise reduction processing on the voice data to obtain corresponding first voice data.

[0186] The second processing unit is configured to perform echo cancellation processing on the first voice data to obtain corresponding second voice data.

[0187] The third processing unit is configured to perform signal enhancement processing on the second voice data to obtain corresponding third voice data.

[0188] The determination unit is configured to determine the third voice data as the target voice data.

[0189] In the embodiment, the operations performed by the above modules or units correspond to the steps of the artificial intelligence-based voice generation method of the foregoing embodiments one by one, and thus will not be described here.

[0190] In some optional implementation manners of the embodiment, the artificial intelligence-based voice generation apparatus further comprises:

[0191] The second acquisition module is configured to acquire an original beam search algorithm.

[0192] The third acquisition module is configured to acquire a preset time repetition perception diversification strategy.

[0193] The optimization module is configured to perform optimization processing on the beam search algorithm based on the time repetition perception diversification strategy to obtain a corresponding optimized algorithm.

[0194] The second determination module is configured to determine the optimized algorithm as the target beam search algorithm.

[0195] In the embodiment, the operations performed by the above modules or units correspond to the steps of the artificial intelligence-based voice generation method of the foregoing embodiments one by one, and thus will not be described here.

[0196] In some optional implementation manners of the embodiment, the artificial intelligence-based voice generation apparatus further comprises:

[0197] The second calling module is configured to call a preset specified storage medium.

[0198] The third determination module is configured to determine, in the specified storage medium, a specified storage location corresponding to voice storage.

[0199] The storage module is configured to store the voice data into the specified storage location in the specified storage medium.

[0200] In the embodiment, the modules or units described above are respectively used to perform operations corresponding to the steps of the artificial intelligence-based speech generation method of the foregoing embodiments, and will not be described here again.

[0201] To solve the above technical problems, the embodiment of the present application further provides a computer device. For details, please refer to Figure 4 , Figure 4 The basic structure block diagram of the computer device of the embodiment is shown in the following figure.

[0202] The computer device 4 includes a memory 41, a processor 42, and a network interface 43, which are connected to each other through a system bus. It should be pointed out that only the computer device 4 with components 41-43 is shown in the figure, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented. Among them, those skilled in the art can understand that the computer device here is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0203] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The computer device can interact with the user through a keyboard, a mouse, a remote controller, a touchpad, a voice control device, and the like.

[0204] The memory 41 includes at least one type of readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit and the external storage device of the computer device 4. In this embodiment, the memory 41 is generally used to store an operating system and various application software installed on the computer device 4, such as computer readable instructions of the artificial intelligence-based speech generation method, etc. In addition, the memory 41 can also be used to temporarily store various data that have been output or will be output.

[0205] The processor 42 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer readable instructions or process data stored in the memory 41, such as computer readable instructions of the artificial intelligence-based speech generation method.

[0206] The network interface 43 can include a wireless network interface or a wired network interface, and is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0207] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0208] In the embodiment of the present application, the input text sequence is first acquired; then a preset text-to-speech model is called; wherein the text-to-speech model comprises a language model and an acoustic model; then the language model is used to decode the text sequence based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set; subsequently, a decoding result is determined from the target candidate sequence set; further, the decoding result is converted into corresponding speech data based on the acoustic model; finally, the speech data is output. The present application realizes efficient and accurate speech generation processing of the text sequence by acquiring the input text sequence, then decoding the text sequence based on the target beam search algorithm and the use of the language model to obtain the target candidate sequence set, determining the decoding result from the target candidate sequence set, and then converting the decoding result into corresponding speech data based on the use of the acoustic model and outputting. The present application effectively reduces the defects and pronunciation errors caused by the decoding method based on sampling in the prior art by adopting the decoding method based on the target beam search algorithm, effectively improving the stability and accuracy of the generated speech data.

[0209] The present application also provides another embodiment, that is, a computer readable storage medium storing computer readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the artificial intelligence-based speech generation method as described above.

[0210] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0211] In the embodiment of the present application, the input text sequence is first acquired; then a preset text-to-speech model is called; wherein the text-to-speech model comprises a language model and an acoustic model; then the language model is used to decode the text sequence based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set; subsequently, a decoding result is determined from the target candidate sequence set; further, the decoding result is converted into corresponding speech data based on the acoustic model; finally, the speech data is output. The present application realizes efficient and accurate speech generation processing of the text sequence by acquiring the input text sequence, then decoding the text sequence based on the target beam search algorithm and the use of the language model to obtain the target candidate sequence set, determining the decoding result from the target candidate sequence set, and then converting the decoding result into corresponding speech data based on the use of the acoustic model and outputting. The present application effectively reduces the defects and pronunciation errors caused by the decoding method based on sampling in the prior art by adopting the decoding method based on the target beam search algorithm, effectively improving the stability and accuracy of the generated speech data.

[0212] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, also can be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application essentially or say the part of the prior art to make contributions can be embodied in the form of software products, the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions to make a terminal device (may be a mobile phone, computer, server, air conditioner, or network equipment, etc.) executes the method described in various embodiments of the present application.

[0213] Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all the embodiments, the preferred embodiments of the present application are given in the drawings, but do not limit the patent scope of the present application. The present application can be realized in many different forms, and contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing specific embodiments, or make equivalent replacement to some technical features. Any equivalent structure made by using the contents of the present application specification and drawings, directly or indirectly used in other related technical fields, are also within the scope of the present application patent protection.

Claims

1. An artificial intelligence-based speech generation method, characterized by, The method comprises the following steps: obtaining an input text sequence; calling a preset text-to-speech model; wherein the text-to-speech model comprises a language model and an acoustic model; decoding the text sequence using the language model based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set; determining a decoding result from the target candidate sequence set; converting the decoding result into corresponding speech data based on the acoustic model; outputting the speech data; wherein the step of decoding the text sequence using the language model based on the preset target beam search algorithm to obtain a corresponding target candidate sequence set specifically comprises: performing candidate sequence expansion processing on the text sequence using the language model based on the target beam search algorithm to obtain a corresponding first candidate sequence; performing repetition detection and penalty processing on the first candidate sequence to obtain a second candidate sequence after score recalculation; performing sequence set selection processing on the second candidate sequence that meets a preset screening condition based on the score of the second candidate sequence to obtain a corresponding first candidate sequence set; repeating the candidate sequence expansion processing, the repetition detection and penalty processing, and the sequence set selection processing at each time step until a preset termination condition is met and a corresponding second candidate sequence set is obtained; taking the second candidate sequence set as the target candidate sequence set. 2.The artificial intelligence-based voice generation method of claim 1, wherein, The step of determining a decoding result from the target candidate sequence set specifically comprises: obtaining the scores of each candidate sequence included in the target candidate sequence set; performing numerical comparison on the scores of all the candidate sequences to select a specified candidate sequence with the highest score from all the candidate sequences; taking the specified candidate sequence as the decoding result. 3.The artificial intelligence-based voice generation method of claim 1, wherein, The step of outputting the speech data specifically comprises: performing quality optimization processing on the speech data based on a preset quality optimization strategy to obtain corresponding target speech data; obtaining a preset specified output mode; outputting the target speech data based on the specified output mode. 4.The AI-based voice generation method of claim 3, wherein, The step of performing quality optimization processing on the speech data based on a preset quality optimization strategy to obtain corresponding target speech data specifically comprises: performing noise reduction processing on the speech data to obtain corresponding first speech data; performing echo cancellation processing on the first speech data to obtain corresponding second speech data; performing signal enhancement processing on the second speech data to obtain corresponding third speech data; taking the third speech data as the target speech data. 5.The artificial intelligence-based voice generation method of claim 1, wherein, Before the step of decoding the text sequence using the language model based on the preset target beam search algorithm to obtain a corresponding target candidate sequence set, the method further comprises: obtaining an original beam search algorithm; obtaining a preset time repetition awareness diversification strategy; optimizing the beam search algorithm based on the time repetition awareness diversification strategy to obtain a corresponding optimized algorithm; taking the optimized algorithm as the target beam search algorithm. 6.The artificial intelligence-based voice generation method of claim 1, wherein, After the step of converting the decoding result into corresponding voice data based on the acoustic model, the method further comprises: calling a preset designated storage medium; determining a designated storage location corresponding to voice storage in the designated storage medium; storing the voice data into the designated storage location in the designated storage medium.

7. An artificial intelligence-based voice generation device, characterized by, The method comprises: a first obtaining module configured to obtain an input text sequence; a first calling module configured to call a preset text-to-voice model; wherein the text-to-voice model comprises a language model and an acoustic model; a decoding module configured to decode the text sequence based on a preset target beam search algorithm using the language model to obtain a corresponding target candidate sequence set; a first determining module configured to determine a decoding result from the target candidate sequence set; a converting module configured to convert the decoding result into corresponding voice data based on the acoustic model; an output module configured to output the voice data. The decoding module comprises: a first processing submodule configured to perform candidate sequence expansion processing on the text sequence based on the target beam search algorithm using the language model to obtain a corresponding first candidate sequence; a second processing submodule configured to perform repetition detection and penalty processing on the first candidate sequence to obtain a second candidate sequence after score recalculation; a third processing submodule configured to perform sequence set selection processing on the second candidate sequence that meets a preset screening condition based on the score of the second candidate sequence to obtain a corresponding first candidate sequence set; a fourth processing submodule configured to repeatedly perform candidate sequence expansion processing, repetition detection and penalty processing, and sequence set selection processing at each time step until a preset termination condition is met and a corresponding second candidate sequence set is obtained; a first determining submodule configured to take the second candidate sequence set as the target candidate sequence set.

8. A computer device, comprising: The method comprises a memory and a processor, the memory stores computer readable instructions, and the processor executes the computer readable instructions to realize the steps of the artificial intelligence-based voice generation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by the processor to realize the steps of the artificial intelligence-based voice generation method according to any one of claims 1 to 6.

Images