Voice generation method and device based on artificial intelligence, computer equipment and medium

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to an artificial intelligence-based voice generation method, which is applied to the field of financial science and technology, and comprises the following steps: obtaining an input text sequence; calling a text-to-speech model; wherein the text-to-speech model comprises a language model and an acoustic model; based on a target beam search algorithm, decoding the text sequence by using the language model to obtain a corresponding target candidate sequence set; determining a decoding result from the target candidate sequence set; converting the decoding result into corresponding voice data based on the acoustic model; and outputting the voice data. The invention further provides a voice generation device based on artificial intelligence, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the voice data can be stored in a block chain. According to the method, the decoding mode based on the target beam search algorithm is adopted, so that the stability and the accuracy of the generated voice data are effectively improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence development technology and the field of financial technology, and in particular to artificial intelligence-based speech generation methods, devices, computer equipment and storage media. Background Art

[0002] In the financial field, with the continuous advancement of technology, Text-To-Speech (TTS) technology has made significant progress. This progress is largely due to the adoption of the TTS paradigm based on the Language Model (LM). LM-based TTS technology achieves a natural conversion from text to speech by simulating a sequence of speech symbols. This conversion process is of great significance for improving the efficiency of financial information transmission and user experience.

[0003] In the LM-based TTS model, pre-trained neural speech taggers play a crucial role. These taggers can quantify speech symbol sequences from continuous speech signals, providing accurate and rich speech data for the TTS model. By training with large speech datasets, the LM-based TTS model can synthesize coherent and natural speech from a single text prompt. This process not only retains the information in the text, but also effectively conveys the sound and emotional characteristics of the original text.

[0004] However, although the LM-based TTS model has achieved remarkable results in synthesizing speech, there are still some technical challenges. Among them, the most prominent is the randomness problem introduced by the sampling-based decoding strategy. In the decoding process, although the introduction of randomness increases the diversity of speech generation to a certain extent, it may also lead to increased instability in the generation of speech symbols. This instability often manifests itself as flaws and pronunciation errors in the speech, which in turn affects the overall quality of the generated speech, resulting in low stability and accuracy of speech generation. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a speech generation method, apparatus, computer device and storage medium based on artificial intelligence to solve the technical problem that the sampling-based decoding strategy adopted by the existing LM-based TTS model in synthesized speech leads to low stability and accuracy of speech generation.

[0006] In order to solve the above technical problems, the present application provides a method for generating speech based on artificial intelligence, which adopts the following technical solutions:

[0007] Get the input text sequence;

[0008] Calling a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model;

[0009] Based on a preset target beam search algorithm, the text sequence is decoded using the language model to obtain a corresponding target candidate sequence set;

[0010] Determining a decoding result from the target candidate sequence set;

[0011] Converting the decoding result into corresponding speech data based on the acoustic model;

[0012] The voice data is output.

[0013] Furthermore, the step of decoding the text sequence using the language model based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set specifically includes:

[0014] Based on the target beam search algorithm, using the language model to perform candidate sequence expansion processing on the text sequence to obtain a corresponding first candidate sequence;

[0015] Performing repeated detection and penalty processing on the first candidate sequence to obtain a second candidate sequence after recalculating the score;

[0016] Based on the score of the second candidate sequence, selecting a sequence set that meets a preset screening condition for the second candidate sequence to obtain a corresponding first candidate sequence set;

[0017] Repeat the candidate sequence expansion process, the repetition detection and penalty process, and the sequence set selection process at each time step until a preset termination condition is met and a corresponding second candidate sequence set is obtained;

[0018] The second candidate sequence set is used as the target candidate sequence set.

[0019] Furthermore, the step of determining a decoding result from the target candidate sequence set specifically includes:

[0020] Obtaining a score for each candidate sequence included in the target candidate sequence set;

[0021] Comparing the scores of all the candidate sequences numerically to select a designated candidate sequence with the highest score from all the candidate sequences;

[0022] The designated candidate sequence is used as the decoding result.

[0023] Furthermore, the step of outputting the voice data specifically includes:

[0024] Performing quality optimization processing on the voice data based on a preset quality optimization strategy to obtain corresponding target voice data;

[0025] Get the preset specified output mode;

[0026] The target voice data is output based on the specified output mode.

[0027] Furthermore, the step of performing quality optimization processing on the voice data based on a preset quality optimization strategy to obtain corresponding target voice data specifically includes:

[0028] Performing noise reduction processing on the voice data to obtain corresponding first voice data;

[0029] Performing echo cancellation processing on the first voice data to obtain corresponding second voice data;

[0030] Performing signal enhancement processing on the second voice data to obtain corresponding third voice data;

[0031] The third voice data is used as the target voice data.

[0032] Furthermore, before the step of decoding the text sequence using the language model based on the preset target beam search algorithm to obtain the corresponding target candidate sequence set, the method further includes:

[0033] Get the original beam search algorithm;

[0034] Get preset time-repeated perception diversification strategies;

[0035] Optimizing the beam search algorithm based on the time repetition perception diversification strategy to obtain a corresponding optimization algorithm;

[0036] The optimization algorithm is used as the target beam search algorithm.

[0037] Further, after the step of converting the decoding result into corresponding speech data based on the acoustic model, the method further includes:

[0038] Call the preset specified storage medium;

[0039] Determining a designated storage location corresponding to the voice storage in the designated storage medium;

[0040] The voice data is stored in the designated storage location in the designated storage medium.

[0041] In order to solve the above technical problems, the embodiment of the present application further provides a speech generation device based on artificial intelligence, which adopts the following technical solution:

[0042] A first acquisition module, used to acquire an input text sequence;

[0043] A first calling module is used to call a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model;

[0044] A decoding module, configured to decode the text sequence using the language model based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set;

[0045] A first determination module, configured to determine a decoding result from the target candidate sequence set;

[0046] A conversion module, used for converting the decoding result into corresponding speech data based on the acoustic model;

[0047] An output module is used to output the voice data.

[0048] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0049] Get the input text sequence;

[0050] Calling a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model;

[0051] Based on a preset target beam search algorithm, the text sequence is decoded using the language model to obtain a corresponding target candidate sequence set;

[0052] Determining a decoding result from the target candidate sequence set;

[0053] Converting the decoding result into corresponding speech data based on the acoustic model;

[0054] The voice data is output.

[0055] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0056] Get the input text sequence;

[0057] Calling a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model;

[0058] Based on a preset target beam search algorithm, the text sequence is decoded using the language model to obtain a corresponding target candidate sequence set;

[0059] Determining a decoding result from the target candidate sequence set;

[0060] Converting the decoding result into corresponding speech data based on the acoustic model;

[0061] The voice data is output.

[0062] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0063] The present application first obtains an input text sequence; then calls a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model; then, based on a preset target beam search algorithm, the language model is used to decode the text sequence to obtain a corresponding target candidate sequence set; subsequently, a decoding result is determined from the target candidate sequence set; further, the decoding result is converted into corresponding speech data based on the acoustic model; and finally, the speech data is output. The present application obtains an input text sequence, then decodes the text sequence based on the target beam search algorithm and the language model to obtain a target candidate sequence set, and determines the decoding result from the target candidate sequence set, and then converts the decoding result into corresponding speech data based on the use of the acoustic model and outputs it, thereby achieving efficient and accurate completion of speech generation processing for the text sequence. The present application adopts a decoding method based on a target beam search algorithm, which effectively reduces the defects and pronunciation errors caused by the sampling-based decoding method in the prior art, and effectively improves the stability and accuracy of the generated speech data. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0065] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0066] Figure 2 A flowchart of an embodiment of a method for speech generation based on artificial intelligence according to the present application;

[0067] Figure 3 is a structural schematic diagram of an embodiment of an artificial intelligence-based speech generation device according to the present application;

[0068] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0070] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0071] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0072] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0073] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0074] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0075] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0076] It should be noted that the artificial intelligence-based speech generation method provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based speech generation device is generally arranged in the server / terminal device.

[0077] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0078] Continue to refer Figure 2 , showing a flow chart of an embodiment of the speech generation method based on artificial intelligence according to the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The speech generation method based on artificial intelligence provided in the embodiment of the present application can be applied to any scenario where speech generation is required, and the speech generation method based on artificial intelligence can be applied to products in these scenarios, for example, speech generation in the field of finance and insurance. The speech generation method based on artificial intelligence comprises the following steps:

[0079] Step S201, obtaining an input text sequence.

[0080] In this embodiment, the electronic device (e.g. Figure 1The server / terminal device shown in the figure) can obtain the image to be checked for duplicates through a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future. The executor of this application is a speech generation system, or simply referred to as the system. The above-mentioned text sequence may be a text input by a user that needs to be converted into text-to-speech processing. The text sequence may be provided by the user, or it may be automatically generated by the system in some way (such as a text generation model). Exemplarily, in the business scenario of finance and insurance, the above-mentioned text sequence may include text data such as transaction text, payment text, and consultation text.

[0081] Step S202, calling a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model.

[0082] In this embodiment, the text-to-speech model may specifically adopt a TTS model based on LM, which includes a trained language model and an acoustic model. The language model is used to provide the probability distribution of the next symbol in the text sequence, while the acoustic model is used to convert the text sequence into speech.

[0083] Step S203 : Based on a preset target beam search algorithm, the language model is used to decode the text sequence to obtain a corresponding target candidate sequence set.

[0084] In this embodiment, the above-mentioned specific implementation process of the preset target beam search algorithm using the language model to decode the text sequence to obtain the corresponding target candidate sequence set will be further described in detail in subsequent specific embodiments of the present application and will not be elaborated here.

[0085] Step S204: determining a decoding result from the target candidate sequence set.

[0086] In this embodiment, the specific implementation process of determining the decoding result from the target candidate sequence set will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0087] Step S205: converting the decoding result into corresponding speech data based on the acoustic model.

[0088] In this embodiment, the decoding result obtained by decoding is passed to the acoustic model. The acoustic model generates corresponding speech features (such as pitch, sound intensity, sound length, etc.) according to the decoding result. And the speech features are converted into actual speech waveforms using speech synthesis technology (such as waveform synthesis, parameter synthesis, etc.), that is, the above-mentioned speech data is obtained.

[0089] Step S206: output the voice data.

[0090] In this embodiment, the specific implementation process of outputting the voice data will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0091] The present application first obtains an input text sequence; then calls a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model; then, based on a preset target beam search algorithm, the language model is used to decode the text sequence to obtain a corresponding target candidate sequence set; subsequently, a decoding result is determined from the target candidate sequence set; further, the decoding result is converted into corresponding speech data based on the acoustic model; and finally, the speech data is output. The present application obtains an input text sequence, then decodes the text sequence based on the target beam search algorithm and the language model to obtain a target candidate sequence set, and determines the decoding result from the target candidate sequence set, and then converts the decoding result into corresponding speech data based on the use of the acoustic model and outputs it, thereby achieving efficient and accurate completion of speech generation processing for the text sequence. The present application adopts a decoding method based on a target beam search algorithm, which effectively reduces the defects and pronunciation errors caused by the sampling-based decoding method in the prior art, and effectively improves the stability and accuracy of the generated speech data.

[0092] In some optional implementations, step S203 includes the following steps:

[0093] Based on the target beam search algorithm, the language model is used to perform candidate sequence expansion processing on the text sequence to obtain a corresponding first candidate sequence.

[0094] In this embodiment, the beam search algorithm is a heuristic search algorithm used to find the optimal or approximately optimal sequence under limited computing resources. It retains a fixed number of candidate sequences (i.e., beam width) and expands these candidate sequences at each step, and finally selects the sequence with the highest score as the result. The above-mentioned target beam search algorithm is an algorithm obtained by optimizing the beam search algorithm using a preset time repetition perception diversification strategy.

[0095] Specifically, load the trained language model, which can predict the probability distribution of the next word or character for a given text sequence. And pre-set the beam width B of the target beam search algorithm. The beam width B is an integer representing the number of highest-scoring candidate sequences to be retained during the beam search process. The selection of the beam width requires a balance between computing resources and decoding quality. And create an empty candidate sequence set corresponding to the above text sequence. At time step t = 0, add a starting symbol (e.g. <s>) as the starting point of the candidate sequence. The above-mentioned candidate sequence expansion process includes: sorting the sequences in the current candidate sequence set by score. Select the first B sequences with the highest scores from the sorted sequences. Then for each selected candidate sequence, use the language model to obtain all possible next tags (words or characters). Traverse these possible tags and generate a new extended sequence for each tag. Subsequently, for each extended sequence, calculate its score. Specifically, the score of the current sequence can be added to the logarithmic probability of the next tag.

[0096] Repeat detection and penalty processing are performed on the first candidate sequence to obtain a second candidate sequence after recalculating the score.

[0097] In this embodiment, the above-mentioned repetition detection and penalty processing includes: using a sliding window technology to detect repeated marks, and punishing the detected repeated marks. Specifically, using the sliding window technology to detect repeated marks includes: defining the length of a sliding window, and for each extended sequence, using the sliding window to compare the marks in the current window with the marks in the previous window to detect time repetition. Punishing the detected repeated marks includes: for each detected repeated mark, modifying its logarithmic probability according to the repetition penalty item. The repetition penalty item includes two hyperparameters (time repetition hyperparameter and inter-beam repetition hyperparameter) for adjusting the penalty strength of repeated marks (respectively controlling the penalty strength of time repetition and inter-beam repetition), and the hyperparameters can be tuned according to experimental data. In addition, for the detected repeated marks, the algorithm will punish their logarithmic probabilities to reduce their scores. Specifically, the penalty calculation method of the modified logarithmic probability of repeated marks = the original logarithmic probability of repeated marks - the time repetition hyperparameter - the inter-beam repetition hyperparameter can be used to perform the penalty processing of the logarithmic probability.

[0098] Based on the score of the second candidate sequence, a sequence set selection process is performed on the second candidate sequence that meets a preset screening condition to obtain a corresponding first candidate sequence set.

[0099] In this embodiment, the sequence set selection process includes: recalculating the score of each extended sequence according to the modified logarithmic probability, then sorting the recalculated scored sequences, and selecting the first B sequences with the highest scores from the sorted sequences as the new candidate sequence set.

[0100] The candidate sequence expansion process, the repetition detection and penalty process, and the sequence set selection process are repeatedly performed at each time step until a preset termination condition is met and a corresponding second candidate sequence set is obtained.

[0101] In this embodiment, the time step t is incremented, and the steps of candidate sequence expansion processing, repeated detection and penalty processing, and sequence set selection processing are repeatedly performed. This process will continue until the end of the sequence is reached (for example, the termination symbol is encountered).< / s> ) or meet other preset termination conditions (such as maximum length limit, decoding time limit, etc.), and obtain the final second candidate sequence set, that is, the target candidate sequence set.

[0102] The second candidate sequence set is used as the target candidate sequence set.

[0103] The present application uses the language model to perform candidate sequence expansion processing on the text sequence based on the target beam search algorithm to obtain the corresponding first candidate sequence; then the first candidate sequence is subjected to repeated detection and penalty processing to obtain the second candidate sequence after the score is recalculated; then based on the score of the second candidate sequence, the second candidate sequence is subjected to sequence set selection processing that meets the preset screening conditions to obtain the corresponding first candidate sequence set; subsequently, the candidate sequence expansion processing, repeated detection and penalty processing, and sequence set selection processing are repeated at each time step until the preset termination condition is met and the corresponding second candidate sequence set is obtained; finally, the second candidate sequence set is used as the target candidate sequence set. The present application can effectively reduce time repetition and repetition between beams by decoding the text sequence based on the target beam search algorithm in conjunction with the language model. This decoding method can encourage the generation of more diverse outputs by penalizing repeated marks while maintaining decoding efficiency, thereby effectively improving the diversity and quality of the generated target candidate sequence set.

[0104] In some optional implementations of this embodiment, step S204 includes the following steps:

[0105] Obtain a score for each candidate sequence included in the target candidate sequence set.

[0106] In this embodiment, the target candidate sequence set includes multiple candidate sequences, and a corresponding score is calculated for each candidate sequence.

[0107] The scores of all the candidate sequences are numerically compared to select a designated candidate sequence with the highest score from all the candidate sequences.

[0108] In this embodiment, the scores of all candidate sequences are numerically compared to obtain a corresponding comparison result, and then the designated score with the highest value can be screened out from all scores according to the comparison result, and the designated candidate sequence corresponding to the designated score can be screened out from all candidate sequences.

[0109] The designated candidate sequence is used as the decoding result.

[0110] The present application obtains the scores of each candidate sequence included in the target candidate sequence set, and then compares the scores of all the candidate sequences numerically to screen out the designated candidate sequence with the highest score from all the candidate sequences; and subsequently uses the designated candidate sequence as the decoding result. The present application obtains the scores of each candidate sequence included in the target candidate sequence set, and then compares the scores of all the candidate sequences numerically to screen out the designated candidate sequence with the highest score from all the candidate sequences and uses it as the final decoding result, thereby effectively ensuring the accuracy of the obtained decoding result.

[0111] In some optional implementations, step S206 includes the following steps:

[0112] The voice data is quality optimized based on a preset quality optimization strategy to obtain corresponding target voice data.

[0113] In this embodiment, the above-mentioned specific implementation process of performing quality optimization processing on the voice data based on a preset quality optimization strategy to obtain the corresponding target voice data will be further described in detail in subsequent specific embodiments of the present application and will not be elaborated on here.

[0114] Get the preset specified output mode.

[0115] In this embodiment, there is no specific limitation on the selection of the above-mentioned designated output mode, which can be determined according to actual business needs, for example, it can include saving the generated voice data as an audio file, or directly playing the voice data on an audio playback device.

[0116] The target voice data is output based on the specified output mode.

[0117] In this embodiment, a certain designated output mode may be used to complete the output processing of the target voice data.

[0118] The present application performs quality optimization processing on the voice data based on a preset quality optimization strategy to obtain corresponding target voice data; then obtains a preset specified output method; and subsequently outputs the target voice data based on the specified output method. After converting the decoding result into corresponding voice data based on the acoustic model, the present application will also automatically and intelligently perform quality optimization processing on the voice data based on the use of the quality optimization strategy to obtain the corresponding target voice data, effectively improving the quality of the generated target voice data. And subsequently, by outputting the target voice data based on the specified output method, the output intelligence of the target voice data is improved, which is conducive to improving the user experience.

[0119] In some optional implementations, performing quality optimization processing on the voice data based on a preset quality optimization strategy to obtain corresponding target voice data includes the following steps:

[0120] The voice data is subjected to noise reduction processing to obtain corresponding first voice data.

[0121] In this embodiment, the noise reduction process for the speech data can be performed by using spectrum subtraction, specifically by analyzing the spectrum of the speech data, estimating the noise spectrum, and subtracting the noise component from the mixed signal to obtain clearer first speech data.

[0122] Perform echo cancellation processing on the first voice data to obtain corresponding second voice data.

[0123] In this embodiment, the echo cancellation process includes: using an adaptive filter, such as a least mean square error (LMS) filter or a normalized least mean square error (NLMS) filter, to distinguish between a direct sound signal and a signal reflected by an environment, thereby minimizing the echo.

[0124] Perform signal enhancement processing on the second voice data to obtain corresponding third voice data.

[0125] In this embodiment, the signal enhancement process includes: using a multi-scale signal analysis method, selecting a suitable wavelet basis function and a threshold to remove noise components and retain the useful part of the signal.

[0126] The third voice data is used as the target voice data.

[0127] The present application performs noise reduction processing on the voice data to obtain the corresponding first voice data; then performs echo cancellation processing on the first voice data to obtain the corresponding second voice data; then performs signal enhancement processing on the second voice data to obtain the corresponding third voice data; and subsequently uses the third voice data as the target voice data. The present application performs noise reduction processing, echo cancellation processing, and signal enhancement processing on the voice data based on the use of a quality optimization strategy, thereby achieving efficient and accurate quality optimization processing of the voice data, and effectively improving the voice quality of the generated target voice data.

[0128] In some optional implementations of this embodiment, before step S203, the electronic device may further perform the following steps:

[0129] Get the raw beam search algorithm.

[0130] In this embodiment, the original beam search algorithm refers to a traditional beam search algorithm. The beam search algorithm is a heuristic search algorithm used to find the optimal or approximately optimal sequence under limited computing resources. It retains a fixed number of candidate sequences (i.e., beam width) and expands these candidate sequences at each step, and finally selects the sequence with the highest score as the result.

[0131] Although the beam search algorithm is designed to identify the highest probability sequence in autoregressive decoding, it faces some challenges when applied to language model-based text-to-speech conversion. These challenges are mainly reflected in two aspects. First, the beam search algorithm may encounter the problem of time series collapse, that is, the generated sequence contains repeated symbols or very short symbol combinations. This causes the generated speech to fall into silence or continue to show artifacts. The initial part shows a recognizable speech pattern corresponding to the speech prompt, while the subsequent generated part consists of noise or silence. This problem may not only appear in greedy decoding, but also occur when the temperature in the sampling strategy is set too low, which reduces randomness and leads to minimized variability in sampling. In addition, the beam search algorithm often faces the challenge of loss of diversity between beams, which means that there is little difference between the token sequences produced by different beams. This leads to almost identical speech content generated by different beams.

[0132] Get the preset time-repetition-aware diversification strategy.

[0133] In this embodiment, the above-mentioned time repetition perception diversification strategy is a strategy composed of time repetition perception, repetition penalty and diversified selection. Specifically, time repetition perception includes: in each step of the decoding process, it will detect whether the current candidate sequence contains a mark that has appeared before. It can be specifically achieved through the sliding window technology, that is, using a window of fixed length to compare the mark in the current window with the mark in the previous window. Repetition penalty includes: for the detected repeated marks, its logarithmic probability is penalized to reduce its score. The size of the penalty term can be controlled by hyperparameters, and these hyperparameters can be tuned according to experimental data. Diversified selection includes: by introducing repetition penalties, the algorithm can encourage the exploration of more different paths during the decoding process, thereby increasing the diversity of decoding results. This helps to avoid falling into a local optimal solution and generate more natural and fluent speech or text.

[0134] The beam search algorithm is optimized based on the time repetition-aware diversification strategy to obtain a corresponding optimization algorithm.

[0135] In this embodiment, in speech generation or natural language processing tasks, although the traditional beam search algorithm can find sequences with higher scores, it sometimes falls into a local optimal solution and generates repeated or redundant content. In order to solve this problem, the present application introduces a temporal repetition perception diversification strategy into the traditional beam search algorithm, thereby obtaining a corresponding optimization algorithm as the final target beam search algorithm.

[0136] The optimization algorithm is used as the target beam search algorithm.

[0137] In this embodiment, the target beam search algorithm obtained can avoid generating repeated or redundant content by introducing a repetition penalty mechanism, thereby improving the quality of the decoding result. The algorithm also encourages the exploration of more different paths during the decoding process, thereby increasing the diversity of the decoding results. In addition, the hyperparameters of the algorithm can be tuned according to experimental data to adapt to different application scenarios and requirements.

[0138] In addition, the processing of the target beam search algorithm includes: at each time step t (starting from 0), the first B sequences with the highest scores are taken from the current candidate sequence set. For each candidate sequence, the language model is used to predict the next possible symbol (word, character or phoneme), and the sequence score after adding the symbol is calculated. Here, the score is usually the cumulative value of the logarithmic probability, which is used to measure the rationality of the sequence. A mechanism of temporal repetition-aware diversity is introduced to detect whether there are temporal repeated symbols in the newly generated sequence, and these repeated symbols are penalized to reduce their scores. This helps to reduce the possibility of generating redundant sequences and improve the diversity of the output. According to the modified score, all extended sequences are reordered, and the first B sequences with the highest scores are selected as the new candidate set for expansion in the next time step. Repeat the above steps until the preset sequence length, decoding time limit or other termination conditions are reached.

[0139] In addition, the target beam search algorithm can be further optimized. For example, a dynamic beam width adjustment mechanism is introduced, and the joint probability of the language model and the acoustic model is considered during the decoding process. Reinforcement learning technology is used to further optimize the decoding strategy, and a policy network is learned to guide the beam search process, thereby improving the decoding efficiency while ensuring the generation quality. With the above improvements, the target beam search algorithm has shown superior performance to the traditional beam search in multiple benchmarks, especially when dealing with long sequences and complex language structures.

[0140] The present application obtains the original beam search algorithm; then obtains the preset time repetition-aware diversification strategy; then optimizes the beam search algorithm based on the time repetition-aware diversification strategy to obtain the corresponding optimization algorithm; and subsequently uses the optimization algorithm as the target beam search algorithm. The present application optimizes the beam search algorithm by introducing the time repetition-aware diversification strategy, and can quickly and accurately construct the corresponding target beam search algorithm, and the constructed target beam search algorithm can effectively improve the decoding quality and increase the decoding diversity. It is beneficial to use the target beam search algorithm and the language model to decode the text sequence in the subsequent process, and can effectively improve the diversity and quality of the generated target candidate sequence set.

[0141] In some optional implementations of this embodiment, after step S205, the electronic device may further perform the following steps:

[0142] Recall the preset specified storage medium.

[0143] In this embodiment, there is no specific limitation on the selection of the above-mentioned designated storage medium. For example, any one of a local database, a disk, a network disk, a blockchain, etc. can be used.

[0144] A designated storage location corresponding to the voice storage is determined in the designated storage medium.

[0145] In this embodiment, the setting of the above-mentioned designated storage location is not specifically limited and can be set according to actual business needs. For example, the above-mentioned designated storage location can be a pre-built dedicated to storing voice data generated during the text-to-speech business process.

[0146] The voice data is stored in the designated storage location in the designated storage medium.

[0147] In this embodiment, the position information (such as path information or coordinate information) of the above-mentioned designated storage location in the designated storage medium can be determined, and then the voice data can be stored in the designated storage location in the designated storage medium according to the position information.

[0148] The present application calls a preset designated storage medium; then determines a designated storage location corresponding to voice storage in the designated storage medium; and subsequently stores the voice data in the designated storage location in the designated storage medium. After converting the decoding result into the corresponding voice data based on the acoustic model, the present application will also intelligently determine a designated storage location corresponding to voice storage in the called designated storage medium, and then store the voice data in the designated storage location in the designated storage medium, thereby effectively ensuring the data security of the voice data and effectively improving the storage intelligence and storage standardization of the voice data.

[0149] In some optional implementations, the user information obtained is subject to the user's consent and complies with relevant laws and policies.

[0150] In addition, the non-Company software tools or components that appear in the embodiments of the present application are merely examples and do not represent actual use.

[0151] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0152] It should be emphasized that in order to further ensure the privacy and security of the above voice data, the above voice data can also be stored in a blockchain node.

[0153] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0154] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0155] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0156] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0157] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0158] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a speech generation device based on artificial intelligence, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0159] like Figure 3 As shown, the speech generation device 300 based on artificial intelligence described in this embodiment includes: a first acquisition module 301, a first calling module 302, a decoding module 303, a first determination module 304, a conversion module 305 and an output module 306. Among them:

[0160] A first acquisition module 301 is used to acquire an input text sequence;

[0161] A first calling module 302 is used to call a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model;

[0162] A decoding module 303 is used to decode the text sequence using the language model based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set;

[0163] A first determination module 304, configured to determine a decoding result from the target candidate sequence set;

[0164] A conversion module 305, configured to convert the decoding result into corresponding speech data based on the acoustic model;

[0165] The output module 306 is used to output the voice data.

[0166] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech generation method in the aforementioned implementation mode, and will not be repeated here.

[0167] In some optional implementations of this embodiment, the decoding module 303 includes:

[0168] A first processing submodule, configured to perform candidate sequence expansion processing on the text sequence using the language model based on the target beam search algorithm to obtain a corresponding first candidate sequence;

[0169] A second processing submodule is used to perform duplicate detection and penalty processing on the first candidate sequence to obtain a second candidate sequence after recalculating the score;

[0170] A third processing submodule, configured to perform sequence set selection processing on the second candidate sequence that meets a preset screening condition based on the score of the second candidate sequence, to obtain a corresponding first candidate sequence set;

[0171] A fourth processing submodule, used to repeatedly perform candidate sequence expansion processing, repetition detection and penalty processing, and sequence set selection processing at each time step until a preset termination condition is met and a corresponding second candidate sequence set is obtained;

[0172] The first determining submodule is configured to use the second candidate sequence set as the target candidate sequence set.

[0173] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech generation method in the aforementioned implementation mode, and will not be repeated here.

[0174] In some optional implementations of this embodiment, the first determining module 304 includes:

[0175] A first acquisition submodule, used to acquire a score of each candidate sequence included in the target candidate sequence set;

[0176] A screening submodule, used for numerically comparing the scores of all the candidate sequences to screen out a designated candidate sequence with the highest score from all the candidate sequences;

[0177] The second determining submodule is configured to use the designated candidate sequence as the decoding result.

[0178] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech generation method in the aforementioned implementation mode, and will not be repeated here.

[0179] In some optional implementations of this embodiment, the output module 306 includes:

[0180] An optimization submodule, used to perform quality optimization processing on the voice data based on a preset quality optimization strategy to obtain corresponding target voice data;

[0181] The second acquisition submodule is used to acquire a preset specified output mode;

[0182] The output submodule is used to output the target voice data based on the specified output mode.

[0183] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech generation method in the aforementioned implementation mode, and will not be repeated here.

[0184] In some optional implementations of this embodiment, the optimization submodule includes:

[0185] A first processing unit, configured to perform noise reduction processing on the voice data to obtain corresponding first voice data;

[0186] A second processing unit, configured to perform echo cancellation processing on the first voice data to obtain corresponding second voice data;

[0187] A third processing unit, configured to perform signal enhancement processing on the second voice data to obtain corresponding third voice data;

[0188] A determining unit is configured to use the third voice data as the target voice data.

[0189] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech generation method in the aforementioned implementation mode, and will not be repeated here.

[0190] In some optional implementations of this embodiment, the artificial intelligence-based speech generation device further includes:

[0191] A second acquisition module, used to acquire the original beam search algorithm;

[0192] The third acquisition module is used to acquire a preset time repetition perception diversification strategy;

[0193] An optimization module, used for optimizing the beam search algorithm based on the time repetition perception diversification strategy to obtain a corresponding optimization algorithm;

[0194] The second determining module is used to use the optimization algorithm as the target beam search algorithm.

[0195] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech generation method in the aforementioned implementation mode, and will not be repeated here.

[0196] In some optional implementations of this embodiment, the artificial intelligence-based speech generation device further includes:

[0197] A second calling module, used for calling a preset designated storage medium;

[0198] A third determining module, configured to determine a designated storage location corresponding to the voice storage in the designated storage medium;

[0199] The storage module is used to store the voice data in the designated storage location in the designated storage medium.

[0200] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech generation method in the aforementioned implementation mode, and will not be repeated here.

[0201] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0202] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0203] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0204] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of a speech generation method based on artificial intelligence, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0205] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions or process data stored in the memory 41, such as computer-readable instructions for running the artificial intelligence-based speech generation method.

[0206] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0207] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0208] In the embodiment of the present application, the present application first obtains the input text sequence; then calls the preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model; then based on the preset target beam search algorithm, the language model is used to decode the text sequence to obtain a corresponding target candidate sequence set; subsequently, the decoding result is determined from the target candidate sequence set; further based on the acoustic model, the decoding result is converted into corresponding speech data; and finally, the speech data is output. The present application obtains the input text sequence, then decodes the text sequence based on the target beam search algorithm and the language model to obtain a target candidate sequence set, and determines the decoding result from the target candidate sequence set, and then converts the decoding result into corresponding speech data based on the use of the acoustic model and outputs it, thereby achieving efficient and accurate completion of the speech generation processing for the text sequence. The present application adopts a decoding method based on the target beam search algorithm, which effectively reduces the defects and pronunciation errors caused by the sampling-based decoding method in the prior art, and effectively improves the stability and accuracy of the generated speech data.

[0209] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned artificial intelligence-based speech generation method.

[0210] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0211] In the embodiment of the present application, the present application first obtains the input text sequence; then calls the preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model; then based on the preset target beam search algorithm, the language model is used to decode the text sequence to obtain a corresponding target candidate sequence set; subsequently, the decoding result is determined from the target candidate sequence set; further based on the acoustic model, the decoding result is converted into corresponding speech data; and finally, the speech data is output. The present application obtains the input text sequence, then decodes the text sequence based on the target beam search algorithm and the language model to obtain a target candidate sequence set, and determines the decoding result from the target candidate sequence set, and then converts the decoding result into corresponding speech data based on the use of the acoustic model and outputs it, thereby achieving efficient and accurate completion of the speech generation processing for the text sequence. The present application adopts a decoding method based on the target beam search algorithm, which effectively reduces the defects and pronunciation errors caused by the sampling-based decoding method in the prior art, and effectively improves the stability and accuracy of the generated speech data.

[0212] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0213] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A speech generation method based on artificial intelligence, characterized in that: The steps include: Get the input text sequence; Calling a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model; Based on a preset target beam search algorithm, the text sequence is decoded using the language model to obtain a corresponding target candidate sequence set; Determining a decoding result from the target candidate sequence set; Converting the decoding result into corresponding speech data based on the acoustic model; The voice data is output.

2. The method for generating speech based on artificial intelligence according to claim 1, characterized in that: The step of decoding the text sequence using the language model based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set specifically includes: Based on the target beam search algorithm, using the language model to perform candidate sequence expansion processing on the text sequence to obtain a corresponding first candidate sequence; Performing repeated detection and penalty processing on the first candidate sequence to obtain a second candidate sequence after recalculating the score; Based on the score of the second candidate sequence, selecting a sequence set that meets a preset screening condition for the second candidate sequence to obtain a corresponding first candidate sequence set; Repeat the candidate sequence expansion process, the repetition detection and penalty process, and the sequence set selection process at each time step until a preset termination condition is met and a corresponding second candidate sequence set is obtained; The second candidate sequence set is used as the target candidate sequence set.

3. The method for generating speech based on artificial intelligence according to claim 1, characterized in that: The step of determining a decoding result from the target candidate sequence set specifically includes: Obtaining a score for each candidate sequence included in the target candidate sequence set; Comparing the scores of all the candidate sequences numerically to select a designated candidate sequence with the highest score from all the candidate sequences; The designated candidate sequence is used as the decoding result.

4. The method for generating speech based on artificial intelligence according to claim 1, characterized in that: The step of outputting the voice data specifically includes: Performing quality optimization processing on the voice data based on a preset quality optimization strategy to obtain corresponding target voice data; Get the preset specified output mode; The target voice data is output based on the specified output mode.

5. The method for generating speech based on artificial intelligence according to claim 4, characterized in that: The step of performing quality optimization processing on the voice data based on a preset quality optimization strategy to obtain corresponding target voice data specifically includes: Performing noise reduction processing on the voice data to obtain corresponding first voice data; Performing echo cancellation processing on the first voice data to obtain corresponding second voice data; Performing signal enhancement processing on the second voice data to obtain corresponding third voice data; The third voice data is used as the target voice data.

6. The method for generating speech based on artificial intelligence according to claim 1, characterized in that: Before the step of decoding the text sequence using the language model based on the preset target beam search algorithm to obtain a corresponding target candidate sequence set, the method further includes: Get the original beam search algorithm; Get preset time-repeated perception diversification strategies; Optimizing the beam search algorithm based on the time repetition perception diversification strategy to obtain a corresponding optimization algorithm; The optimization algorithm is used as the target beam search algorithm.

7. The method for generating speech based on artificial intelligence according to claim 1, characterized in that: After the step of converting the decoding result into corresponding speech data based on the acoustic model, the method further includes: Call the preset specified storage medium; Determining a designated storage location corresponding to the voice storage in the designated storage medium; The voice data is stored in the designated storage location in the designated storage medium.

8. A speech generation device based on artificial intelligence, characterized in that: include: A first acquisition module, used to acquire an input text sequence; A first calling module is used to call a preset text-to-speech model; wherein the text-to-speech model includes a language model and an acoustic model; A decoding module, configured to decode the text sequence using the language model based on a preset target beam search algorithm to obtain a corresponding target candidate sequence set; A first determination module, configured to determine a decoding result from the target candidate sequence set; A conversion module, used for converting the decoding result into corresponding speech data based on the acoustic model; An output module is used to output the voice data.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the artificial intelligence-based speech generation method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the artificial intelligence-based speech generation method as described in any one of claims 1 to 7 are implemented.

Images