Artificial intelligence-based audio generation method, apparatus, device, and storage medium

Abstract

The application relates to the field of artificial intelligence and discloses an audio generation method, device and equipment based on artificial intelligence and a storage medium, the method comprising the following steps: obtaining to-be-converted audio data, and obtaining a target field identifier corresponding to expected sound quality and / or expected emotion; performing cepstrum feature conversion based on a speech spectrum on the to-be-converted audio data to obtain cepstrum feature data of the to-be-converted audio data; inputting the cepstrum feature data and the target field identifier into a speech conversion model to encode the cepstrum feature data at an encoding layer, randomly sample the encoded vector at a resampling layer, and decode and reconstruct the vector sampled at a decoding layer based on the target field identifier to obtain target audio data; and on the basis of effectively removing sound rhythm information of a sounder, the application effectively converts the audio data into audio data with expected rhythm style information, thereby improving an audio conversion effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to an audio generation method and device based on artificial intelligence, an audio generation equipment and a storage medium. BACKGROUND

[0002] A speech synthesis system is an indispensable part of an intelligent dialogue system. Academia and industry try to synthesize real human voices with limited resources and time, so that the sound quality of the synthesized voice matches the target. Due to the initial speech synthesis performance, a speech conversion model based on a neural network has been widely used in speech synthesis systems.

[0003] However, the existing speech conversion model still has defects. In the process of converting the input source audio, it is difficult to separate the rhythm information of the source audio from the source audio, so that the converted target audio still carries the rhythm characteristics of the source audio, resulting in poor audio conversion effect. SUMMARY

[0004] The present application provides an audio generation method and device based on artificial intelligence, and an audio generation equipment and a storage medium, to solve the technical problem that it is difficult to separate the rhythm information of the source audio from the source audio in the existing speech conversion process, resulting in poor audio conversion effect.

[0005] An audio generation method based on artificial intelligence is provided, comprising:

[0006] Obtaining audio data to be converted, and obtaining a target domain identifier corresponding to the expected audio quality and / or the expected emotion;

[0007] Obtaining a speech conversion model, the speech conversion model being a deep learning model obtained by training different audio data in advance, the speech conversion model comprising an encoding layer, a resampling layer and a decoding layer;

[0008] Converting the audio data to be converted based on the cepstrum feature of the speech spectrum to obtain the cepstrum feature data of the audio data to be converted;

[0009] Inputting the cepstrum feature data and the target domain identifier into the speech conversion model to encode the cepstrum feature data in the encoding layer, randomly sampling the encoded vector in the resampling layer, and decoding and reconstructing the vector obtained by sampling based on the target domain identifier in the decoding layer to obtain target audio data.

[0010] Further, converting the audio data to be converted based on the cepstrum feature of the speech spectrum to obtain the cepstrum feature data of the audio data to be converted, comprising:

[0011] Mel-spectrum conversion is performed on the audio data to be converted to obtain the Mel-spectrum of the audio data to be converted;

[0012] performing cepstrum analysis on the mel-frequency spectrum of the audio data to be converted to obtain mel-frequency cepstral coefficients of the audio data to be converted;

[0013] using the mel-frequency cepstral coefficients of the audio data to be converted as the cepstral feature data.

[0014] Further, performing mel-frequency spectrum conversion on the audio data to be converted to obtain a mel-frequency spectrum of the audio data to be converted, comprising:

[0015] performing pre-emphasis processing on the audio data to be converted to obtain pre-emphasis data;

[0016] performing frame processing on the pre-emphasis data, and performing windowing processing on each frame of pre-emphasis data obtained by the frame processing to obtain audio data to be processed;

[0017] performing Fourier transform on the audio data to be processed to obtain a power spectrum of the audio data to be converted;

[0018] inputting the power spectrum of the audio data to be converted into a mel-scale-based triangular filter to perform multiple filtering to obtain a mel-frequency spectrum of the audio data to be converted.

[0019] Further, encoding the cepstral feature data at an encoding layer, randomly sampling the encoded vector at a resampling layer, and decoding and reconstructing the vector obtained by the sampling based on a target domain identifier at a decoding layer to obtain target audio data, comprising:

[0020] inputting the cepstral feature data into the encoding layer to perform variational auto-encoding to obtain a hidden state feature vector;

[0021] inputting the hidden state feature vector into the resampling layer to perform random sampling to obtain a resampled feature vector;

[0022] inputting the resampled feature vector into the decoding layer to perform decoding, and constructing speech based on the target domain identifier to obtain the target audio data.

[0023] Further, inputting the cepstral feature data into the encoding layer to perform variational auto-encoding to obtain a hidden state feature vector, comprising:

[0024] inputting the cepstral feature data into an encoder of the encoding layer, and performing variational auto-encoding on the cepstral feature data by the encoder to obtain an encoded feature vector of the cepstral feature data;

[0025] sampling the encoded feature vector of the cepstral feature data based on a normal distribution to obtain the hidden state feature vector.

[0026] Further, the resampling feature vector is input into a decoding layer for decoding, and a target audio data is obtained by voice construction based on the target domain identifier, comprising:

[0027] The resampling feature vector is input into a decoder of the decoding layer, and the resampling feature vector is decoded by the decoder to obtain decoded feature data;

[0028] The target domain identifier and the decoded feature data are input into a flow module of the decoding layer for voice construction to obtain the target audio data.

[0029] Further, the resampling feature vector is obtained by the following way:

[0030] A random sampling function is obtained, and a random threshold of each frame vector in the hidden state feature vector is determined according to the random sampling function;

[0031] According to the size of the random threshold and the preset value, each frame vector of the hidden state feature vector is upsampled or downsampled to obtain the resampling feature vector.

[0032] An audio generation method based on artificial intelligence is provided, comprising:

[0033] A first obtaining module is configured to obtain to-be-converted audio data, and obtain a target domain identifier corresponding to an expected voice quality and / or an expected emotion;

[0034] A second obtaining module is configured to obtain a voice conversion model, wherein the voice conversion model is a deep learning model obtained by pre-training according to different audio data, and the voice conversion model comprises an encoding layer, a resampling layer, and a decoding layer;

[0035] A feature conversion module is configured to perform a voice spectrum-based cepstrum feature conversion on the to-be-converted audio data to obtain cepstrum feature data of the to-be-converted audio data;

[0036] A voice conversion module is configured to input the cepstrum feature data and the target domain identifier into the voice conversion model, encode the cepstrum feature data in the encoding layer, randomly sample the encoded vector in the resampling layer, and decode and reconstruct the vector obtained by sampling in the decoding layer based on the target domain identifier to obtain target audio data.

[0037] A computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned audio generation method based on artificial intelligence when executing the computer program.

[0038] A computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the above-mentioned audio generation method based on artificial intelligence.

[0039] In one scheme provided by the artificial intelligence-based audio generation method, device, equipment and storage medium, the audio data to be converted is obtained, and a target domain identifier corresponding to an expected audio quality and / or an expected emotion is obtained. A speech conversion model is obtained, the speech conversion model being a deep learning model obtained by pre-training according to different audio data, the speech conversion model including an encoding layer, a resampling layer and a decoding layer. The audio data to be converted is subjected to cepstrum feature conversion based on a speech spectrum to obtain cepstrum feature data of the audio data to be converted. The cepstrum feature data and the target domain identifier are input into the speech conversion model to encode the cepstrum feature data at the encoding layer, randomly sample the encoded vector at the resampling layer, and decode and reconstruct the vector sampled at the decoding layer based on the target domain identifier to obtain target audio data. In the present application, the cepstrum feature data subjected to cepstrum feature conversion is used as the input of the speech conversion model, and contains less pitch information. In the speech conversion process, the random sampling of the encoded vector at the resampling layer can effectively remove the rhythm information of the sounder. Finally, the speech is reconstructed based on the expected audio quality and / or the expected emotion corresponding to the target domain identifier. On the basis of effectively removing the sounder's voice rhythm information, the audio data with the expected rhythm style information is effectively converted, and the audio conversion effect is improved. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0041] Figure 1 is an application environment schematic diagram of the artificial intelligence-based audio generation method in an embodiment of the present application;

[0042] Figure 2 is a flowchart of the artificial intelligence-based audio generation method in an embodiment of the present application;

[0043] Figure 3 is Figure 2 is an implementation flowchart of step S30 in

[0044] Figure 4 is Figure 3 is an implementation flowchart of step S31 in

[0045] Figure 5 is Figure 2 is an implementation flowchart of step S40 in

[0046] Figure 6 is Figure 5 An implementation flowchart of step S41 in the method is shown in FIG. 4.

[0047] Figure 7 is Figure 5 An implementation flowchart of step S42 in the method is shown in FIG. 5.

[0048] Figure 8 is Figure 5 An implementation flowchart of step S43 in the method is shown in FIG. 6.

[0049] Figure 9 is a structural diagram of an audio generation device based on artificial intelligence in an embodiment of the present application.

[0050] Figure 10 is a structural diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0052] The audio generation method based on artificial intelligence provided by the embodiments of the present application can be applied in an application environment such as Figure 1 wherein the terminal device communicates with the server through a network.

[0053] When a voice conversion task needs to be performed, the server acquires the audio data to be converted sent by the user through the terminal device, and determines the corresponding target domain identifier according to the expected voice quality information and / or emotion information to be converted. Each target domain identifier corresponds to an expected voice quality and / or an expected emotion. Then the server acquires a voice conversion model, which is a deep learning model obtained by pre-training according to different audio data. The voice conversion model includes an encoding layer, a resampling layer, and a decoding layer, and performs cepstrum feature conversion based on the voice spectrum on the audio data to be converted to obtain cepstrum feature data of the audio data to be converted. Then the cepstrum feature data and the target domain identifier are input into the voice conversion model to encode the cepstrum feature data in the encoding layer, randomly sample the encoded vector in the resampling layer, and decode and reconstruct the vector sampled in the decoding layer based on the target domain identifier to obtain target audio data. In this embodiment, the cepstrum feature data converted by the cepstrum feature conversion is used as the input of the voice conversion model, which contains less pitch information. In the voice conversion process, the random sampling of the encoded vector in the resampling layer can effectively remove the rhythm information of the sounder. Finally, the voice is reconstructed based on the expected voice quality and / or expected emotion corresponding to the target domain identifier. On the basis of effectively removing the sounder's voice rhythm information, the audio data with the expected rhythm style information is effectively converted, the audio conversion effect is improved, and the artificial intelligence of the voice system is improved. In addition, the audio generation method in this embodiment does not depend on text, and does not need to be annotated in the voice conversion process, that is, the rhythm can be effectively converted, especially in terms of rhythm, to match the target domain style. At the same time, the rhythm style information is effectively converted only in the case of providing a given high-level global domain summary (i.e., the target domain identifier, which can be a user identity identifier for the expected converted voice quality and / or an emotion category identifier), which is simple and convenient, can reduce the amount of data processing.

[0054] Among them, the voice conversion model, the target domain identifier and other data are stored in the database of the server, so that when the voice conversion task needs to be performed, the relevant data can be directly acquired according to the needs to reduce the data processing time, thereby improving the prediction efficiency.

[0055] The database in the embodiment is stored in a blockchain network, and is used to store data used in and generated by the artificial intelligence-based audio generation method, such as a target voice conversion model, a target domain identifier, and the like. The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. The blockchain is essentially a decentralized database, and is a series of data blocks associated using cryptographic methods, each of which contains information of a batch of network transactions, and is used to verify the validity (anti-fake) of the information and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer, and the like. Deploying the database in the blockchain can improve the security of data storage.

[0056] The terminal device can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0057] In an embodiment, as shown in Figure 2 , an artificial intelligence-based audio generation method is provided. Taking a server in Figure 1 as an example, the method includes the following steps:

[0058] S10: Obtain audio data to be converted, and obtain a target domain identifier corresponding to a desired voice quality and / or a desired emotion.

[0059] When a voice conversion task needs to be performed, the server obtains audio data to be converted, that is, source audio data that needs to be converted into a desired voice quality and / or a desired emotion, determines a desired voice quality and / or a desired emotion, and then obtains a target domain identifier corresponding to the desired voice quality and / or the desired emotion.

[0060] Each domain identifier corresponds to a desired voice quality and / or a desired emotion, that is, a domain identifier corresponds to a desired voice quality, or a domain identifier corresponds to a desired emotion, or a domain identifier corresponds to a voice information group composed of a desired voice quality and a desired emotion. The domain identifier can be an identity identifier of a source user of the desired voice quality, can be an emotion category symbol of the desired emotion, or can be a unique identifier of the voice information group.

[0061] The domain identifier and the corresponding desired voice quality and / or desired emotion are kept in the database and are one-to-one bound, so as to directly determine the domain identifier according to the voice conversion expectation in the subsequent process.

[0062] S20: Obtain a voice conversion model, the voice conversion model including an encoding layer, a resampling layer, and a decoding layer.

[0063] While acquiring the audio data to be converted, the server also needs to acquire a voice conversion model. The voice conversion model is a deep learning model obtained by pre-training according to different audio data. The voice conversion model includes an encoding layer, a resampling layer, and a decoding layer.

[0064] The training process of the voice conversion model is as follows:

[0065] S01: Acquire a plurality of sample audios, and determine the domain identifier corresponding to the expected voice quality and / or expected emotion to be converted. Each sample audio corresponds to a standard audio in which a target user adopts an expected voice quality and / or an expected emotion to perform real voice conversion on the sample audio.

[0066] S02: Acquire a preset model containing initial parameters, which includes an encoding layer, a resampling layer, and a decoding layer.

[0067] S03: Perform cepstrum feature conversion based on the voice spectrum on the sample audio to obtain the cepstrum feature of the sample audio.

[0068] First, the sample audio needs to be converted into a mel-frequency spectrum to obtain the mel-frequency spectrum of the audio data to be converted. Then, the mel-frequency spectrum of the sample audio is analyzed to obtain the mel-frequency cepstrum coefficient of the sample audio as the cepstrum feature of the sample audio. The process of obtaining the cepstrum feature of the sample audio can refer to the specific process of obtaining the cepstrum feature data of the audio data to be converted in the subsequent processing of the audio data to be converted, which will not be repeated here.

[0069] S04: Input the cepstrum feature of the sample audio and the corresponding domain identifier into the preset model, encode it through the encoding layer, sample it through the resampling layer, and reconstruct the voice based on the output of the resampling layer and the corresponding domain identifier in the decoder to obtain the predicted audio.

[0070] S05: Determine the total loss value between the predicted audio and the standard audio.

[0071] S06: When the total loss value does not satisfy the convergence condition, iteratively update the parameters of the preset model according to the plurality of sample audios until the total loss value satisfies the convergence condition, and output the preset model that satisfies the convergence condition as the voice conversion model.

[0072] S30: Perform cepstrum feature conversion based on the voice spectrum on the audio data to be converted to obtain the cepstrum feature data of the audio data to be converted.

[0073] After acquiring the audio data to be converted, perform cepstrum feature conversion based on the voice spectrum on the audio data to be converted to obtain the cepstrum feature data of the audio data to be converted.

[0074] The audio data to be converted needs to be subjected to spectrum extraction processing to obtain a speech spectrum of the audio data to be converted, and then a cepstrum analysis is performed based on the speech spectrum of the audio data to be converted to obtain the cepstrum feature data of the audio data to be converted. After the cepstrum feature conversion of the audio data to be converted based on the speech spectrum, the cepstrum feature data of the audio data to be converted carries a small amount of pitch information, and compared with the traditional model input of the frequency of the audio data to be converted, the influence of the voice rhythm of the original singer on the subsequent converted audio is reduced.

[0075] S40: input the cepstrum feature data and the target domain identifier into the speech conversion model to encode the cepstrum feature data in the encoding layer, randomly sample the encoded vector in the resampling layer, and decode and reconstruct the sampled vector based on the target domain identifier in the decoding layer to obtain the target audio data.

[0076] After obtaining the cepstrum feature data of the audio data to be converted, the cepstrum feature data with a small amount of pitch information and the target domain identifier are input into the speech conversion model to encode the cepstrum feature data in the encoding layer, randomly sample the encoded vector in the resampling layer, and decode and reconstruct the sampled vector based on the target domain identifier in the decoding layer to obtain the target audio data.

[0077] In this embodiment, the audio data to be converted is obtained, and the audio quality information and / or emotional information expected to be converted is obtained to determine the corresponding target domain identifier, and then a speech conversion model is obtained, the speech conversion model being a deep learning model obtained by pre-training according to different audio data, the speech conversion model including an encoding layer, a resampling layer and a decoding layer, and performing cepstrum feature conversion based on a speech spectrum on the audio data to be converted to obtain cepstrum feature data of the audio data to be converted, and inputting the cepstrum feature data and the target domain identifier into the speech conversion model to encode the cepstrum feature data in the encoding layer, randomly sample the encoded vector in the resampling layer, and decode and reconstruct the vector sampled in the decoding layer based on the target domain identifier to obtain target audio data; in this embodiment, the cepstrum feature data obtained by cepstrum feature conversion is used as the input of the speech conversion model, which contains less pitch information, and in the speech conversion process, the rhythm information of the sounder can be effectively removed by randomly sampling the encoded vector in the resampling layer, and finally the speech is reconstructed based on the expected audio quality and / or expected emotion corresponding to the target domain identifier, which effectively converts the audio data with expected rhythm style information on the basis of effectively removing the sounder's voice rhythm information, improves the audio conversion effect, and improves the artificial intelligence of the speech system. In addition, the audio generation method in this embodiment does not depend on text, and does not need to be annotated in the speech conversion process, that is, the rhythm can be effectively converted, especially in terms of rhythm, to match the target domain style, and at the same time, the rhythm style information is effectively converted only under the condition of providing a given high-level global domain summary (i.e., the target domain identifier, which can be a user identifier for expected conversion of voice quality and / or an emotional category identifier), which is simple and convenient, and can reduce the amount of data processing.

[0078] In one embodiment, as shown in FIG. 3, in step S30, the audio data to be converted is converted based on the speech spectrum to obtain the cepstrum feature data of the audio data to be converted. Specifically, the following steps are included: Figure 3

[0079] S31: Perform Mel spectrum conversion on the audio data to be converted to obtain the Mel spectrum of the audio data to be converted.

[0080] After obtaining the audio data to be converted, the Mel spectrum conversion is performed on the audio data to be converted to obtain the Mel spectrum of the audio data to be converted, that is, the Mel filter is used to filter the audio data to be converted to obtain the Mel spectrum of the converted audio data.

[0081] S32: Perform cepstrum analysis on the Mel spectrum of the audio data to be converted to obtain the Mel frequency cepstrum coefficient of the audio data to be converted.

[0082] ​S33: taking the mel-frequency cepstral coefficients of the audio data to be converted as the cepstral feature data.

[0083] After obtaining the mel-frequency spectrum of the audio data to be converted, the mel-frequency spectrum of the audio data to be converted is subjected to cepstral analysis to obtain the mel-frequency cepstral coefficients of the audio data to be converted, and finally the mel-frequency cepstral coefficients of the audio data to be converted are taken as the cepstral feature data.

[0084] The cepstral analysis includes taking a logarithm and performing an inverse transform. After performing the inverse transform, a preset number of mel-frequency cepstral coefficients (such as the 2nd to 13th mel-frequency cepstral coefficients) of the inverse transform can be taken as the cepstral feature data, so that the low-frequency part of the mel-frequency cepstral coefficients is taken as the cepstral feature data, thereby improving the accuracy of the cepstral feature data.

[0085] The mel-frequency spectrum of the audio data to be converted is subjected to cepstral analysis, and the process details are as follows: the mel-frequency spectrum of the audio data to be converted is subjected to convolution processing to obtain convolution data, the mel-frequency spectrum of the audio data to be converted is subjected to envelope extraction to obtain a spectrum envelope, the mel-frequency spectrum of the audio data to be converted is subjected to logarithm processing based on the spectrum envelope to obtain logarithm data, the logarithm data is subjected to inverse transform to obtain mel-frequency cepstral coefficients restored as convolution signals, and a preset number of mel-frequency cepstral coefficients (such as the 2nd to 13th mel-frequency cepstral coefficients) are taken as the cepstral feature data.

[0086] The audio signal is a convolution signal, the first step is to convert it into a multiplicative signal through convolution (convolution in time domain is equivalent to multiplication in frequency domain), the second step is to convert the multiplicative signal into an additive signal through logarithm taking, and the third step is to perform inverse transform to restore it to a convolution signal. At this time, although the front and back are both time domain sequences, they are different in discrete time domain, so the latter is called cepstral frequency domain. Since the peak value in the speech spectrum represents the main frequency component of the speech, these peak values are called formants, and the formants carry the sound recognition attribute. Therefore, after obtaining the mel-frequency spectrum of the audio data to be converted, a spectrum envelope needs to be extracted based on the mel-frequency spectrum and the conversion process, and the spectrum envelope is a smooth curve connecting these formant points.

[0087] In this embodiment, the Mel-frequency spectrum conversion is performed on the audio data to be converted to obtain the Mel-frequency spectrum of the audio data to be converted, and then the cepstrum analysis is performed on the Mel-frequency spectrum of the audio data to be converted to obtain the Mel-frequency cepstrum coefficient of the audio data to be converted. The Mel-frequency cepstrum coefficient of the audio data to be converted is taken as the cepstrum feature data, which refines the specific steps of performing the cepstrum feature conversion based on the speech spectrum on the audio data to be converted. The Mel-frequency filtering conversion is performed on the audio data to be converted, so that the cepstrum feature data is obtained based on the Mel-frequency spectrum, which can make the cepstrum feature data closer to the hearing habits of human ears. In other embodiments, the audio data to be converted can also be filtered by other filtering methods to obtain the cepstrum feature data through the cepstrum analysis. For example, the Bessel filtering method, the linear filtering method, etc.

[0088] In an embodiment, as shown in FIG. 3, Figure 4 the step S31, i.e., the Mel-frequency spectrum conversion is performed on the audio data to be converted to obtain the Mel-frequency spectrum of the audio data to be converted, specifically includes the following steps:

[0089] S311: Pre-emphasis processing is performed on the audio data to be converted to obtain pre-emphasis data.

[0090] After obtaining the audio data to be converted, the pre-emphasis processing is performed on the audio data to be converted to obtain the pre-emphasis data. The pre-emphasis process is to pass the audio data to be converted through a high-pass filter to improve the high-frequency part of the audio data to be converted, eliminate the effects of vocal cords and lips in the sound production process, and compensate for the high-frequency part of the audio signal suppressed by the sound production system to highlight the resonance peak of the high frequency.

[0091] S312: The pre-emphasis data is subjected to frame processing, and each frame of pre-emphasis data obtained by the frame processing is subjected to windowing processing to obtain the audio data to be processed.

[0092] After the pre-emphasis processing is performed on the audio data to be converted to obtain the pre-emphasis data, the pre-emphasis data is subjected to frame processing, and each frame of pre-emphasis data obtained by the frame processing is subjected to windowing processing to obtain the audio data to be processed.

[0093] Wherein, the frame is to be converted audio data of N sample points set as a unit, called frame, N generally takes 256 or 512 or 1024, etc., usually cover 20-30 ms, at the same time, in order to keep continuity between frames, prevent the change between two frames too big, two adjacent frames will have an overlap area, this overlap area contains M sample points, generally M is 1 / 2 or 1 / 3 of N. Such as 8 kHz audio data, if the frame length is 256, the corresponding time length is 256 / 8000*1000 = 32 ms. Windowing is to multiply the pre-emphasis data of each frame obtained by frame by Hamming window, in order to increase the continuity of the left and right ends of the frame, so as to carry out Fourier transform transform subsequently.

[0094] S313: Fourier transform is carried out on the audio data to be processed, and the power spectrum of the audio data to be converted is obtained.

[0095] After obtaining the audio data to be processed, Fourier transform (fast Fourier transform) is carried out on the audio data to be processed, and the power spectrum of the audio data to be converted is obtained. Because the signal characteristics are difficult to distinguish in time domain, the time domain energy is converted into frequency domain energy distribution by Fourier transform.

[0096] S314: The power spectrum of the audio data to be converted is input into the triangular filter based on mel scale for multiple filtering, and the mel spectrum of the audio data to be converted is obtained.

[0097] After Fourier transform is carried out on the audio data to be processed, and the power spectrum of the audio data to be converted is obtained, the power spectrum of the audio data to be converted is input into the triangular filter based on mel scale for multiple filtering, and the mel spectrum of the audio data to be converted is obtained.

[0098] Wherein, the triangular filter is a filter bank based on mel scale, which is a filter bank with K filters, and the center frequency of the filter is f(m), m = 1, 2, …, K, wherein K generally takes 22-26, and the interval between the center frequencies of the filters increases with the increase of m.

[0099] In this embodiment, by pre-emphasis processing on the audio data to be converted, pre-emphasis data is obtained, then the pre-emphasis data is frame processed, and each frame of pre-emphasis data obtained by frame is windowed to obtain the audio data to be processed, then Fourier transform is carried out on the audio data to be processed, and the power spectrum of the audio data to be converted is obtained, finally the power spectrum of the audio data to be converted is input into the triangular filter based on mel scale for multiple filtering, and the mel spectrum of the audio data to be converted is obtained, which clearly shows the specific process of converting the audio data to be converted into mel spectrum, ensures the accuracy of the mel spectrum obtained after a series of processing, and provides a basis for subsequent speech conversion.

[0100] In an embodiment, as shown in Figure 5 In step S40, i.e., in the encoding layer, the cepstrum feature data is encoded, in the resampling layer, the encoded vector is randomly sampled, and in the decoding layer, the sampled vector is decoded and reconstructed based on the target domain identifier to obtain the target audio data, specifically including the following steps:

[0101] S41: input the cepstrum feature data into the encoding layer for variational auto-encoding to obtain a hidden state feature vector.

[0102] After obtaining the cepstrum feature data of the audio data to be converted, the cepstrum feature data of the audio data to be converted is input into the encoding layer for variational auto-encoding to obtain a hidden state feature vector.

[0103] In this embodiment, the encoding layer is a structure based on a variational auto-encoder. It should be understood that a variational auto-encoder is a structure composed of an encoder and a decoder, which is trained to minimize the error between the data output by the encoding layer and the initial data.

[0104] In other embodiments, the principle of the encoder of the encoding layer can also be other, which will not be repeated here.

[0105] S42: input the hidden state feature vector into the resampling layer for random sampling to obtain a resampled feature vector.

[0106] After inputting the cepstrum feature data into the encoding layer for variational auto-encoding to obtain a hidden state feature vector, the hidden state feature vector is input into the resampling layer for random sampling to obtain a resampled feature vector.

[0107] S43: input the resampled feature vector into the decoding layer for decoding, and based on the target domain identifier, the decoding result is constructed into speech to obtain the target audio data.

[0108] After inputting the hidden state feature vector into the resampling layer for random sampling to obtain a resampled feature vector, the resampled feature vector is input into the decoding layer for decoding, and based on the target domain identifier, the decoding result is constructed into speech to obtain the target audio data.

[0109] Since the encoding layer in the embodiment is a structure based on a variational autoencoder, in the process of inputting the cepstrum feature data into the encoding layer for variational autoencoding, and based on the normal distribution of the cepstrum feature data, the hidden state feature vector is obtained only once, and then the random sampling in the resampling layer is performed, which is a second sampling of the cepstrum feature data, so that the resampled feature vector greatly eliminates the prosody information of the original speaker, and the random sampling in the resampling layer makes the lengths of the collected audio signal segments inconsistent, further eliminating the prosody information of the original speaker, so that the subsequently reconstructed target audio information is closer to the expected style.

[0110] In the embodiment, the cepstrum feature data is input into the encoding layer for variational autoencoding to obtain a hidden state feature vector, and the hidden state feature vector is input into the resampling layer for random sampling to obtain a resampled feature vector, and then the resampled feature vector is input into the decoding layer for decoding, and the decoding result is constructed into target audio data based on the target domain identifier, which refines the steps of encoding the cepstrum feature data in the encoding layer, randomly sampling the encoded vector in the resampling layer, and decoding and reconstructing the vector sampled based on the target domain identifier in the decoding layer to obtain the target audio data. In the process of voice conversion, the cepstrum feature data is sampled once by the variational autoencoding of the encoding layer, and then the resampling layer is sampled twice, which greatly eliminates the prosody information of the original speaker, and the random sampling in the resampling layer makes the lengths of the collected audio signal segments inconsistent, further eliminating the prosody information of the original speaker, so that the subsequently reconstructed target audio information is closer to the expected style, greatly eliminating the prosody information of the original speaker, so that the subsequently reconstructed target audio information is closer to the expected style, and the voice conversion effect is effectively improved.

[0111] In an embodiment, as shown in Figure 6 the step S41, that is, the cepstrum feature data is input into the encoding layer for variational autoencoding to obtain a hidden state feature vector, specifically including the following steps:

[0112] S411: input the cepstrum feature data into the encoder of the encoding layer, and perform variational autoencoding on the cepstrum feature data by the encoder to obtain an encoded feature vector of the cepstrum feature data;

[0113] S412: sample the encoded feature vector of the cepstrum feature data based on the normal distribution to obtain a hidden state feature vector.

[0114] After obtaining the cepstrum feature data of the audio data to be converted, the cepstrum feature data is input into the encoder of the encoding layer, and the cepstrum feature data is variational autoencoded by the encoder to obtain an encoded feature vector of the cepstrum feature data, and then the encoded feature vector of the cepstrum feature data is sampled based on the normal distribution to obtain a hidden state feature vector.

[0115] In this embodiment, by inputting the cepstrum feature data into the encoder of the encoding layer, the cepstrum feature data is variational self-encoded by the encoder to obtain an encoding feature vector of the cepstrum feature data, and then the encoding feature vector of the cepstrum feature data is sampled based on a normal distribution to obtain a hidden state feature vector. The specific process of inputting the cepstrum feature data into the encoding layer for variational self-encoding to obtain the hidden state feature vector is clarified, which improves the basis for subsequent processing.

[0116] In an embodiment, as shown in FIG. 4B, the resampled feature vector in step S42 is obtained in the following manner: Figure 7

[0117] S421: Obtain a random sampling function, and determine a random threshold of each frame vector in the hidden state feature vector according to the random sampling function.

[0118] S422: According to the size of the random threshold and a preset value, up-sample or down-sample each frame vector of the hidden state feature vector to obtain the resampled feature vector.

[0119] After the hidden state feature vector is input into the resampling layer, the resampling layer performs the following operations: obtaining a random sampling function, and determining a random threshold of each frame vector in the hidden state feature vector according to the random sampling function; according to the size of the random threshold and a preset value, up-sampling or down-sampling each frame vector of the hidden state feature vector to obtain the resampled feature vector. Since the threshold of each frame vector is random, some parts of the speech can be down-sampled, while other parts can be up-sampled, thereby improving the randomness of sampling and eliminating the prosody information of the original speaker.

[0120] The resampling algorithm is as follows:

[0121] Determine the random threshold τ(t) of the current frame vector in the hidden state feature vector, and determine whether the random threshold τ(t) of the current frame vector is less than a preset value (such as 1):

[0122] If the random threshold τ(t) is less than the preset value, the current frame vector is merged into the previous audio segment, or the current frame vector is generated as a new audio segment, thereby realizing down-sampling. When the similarity between the current frame vector and the previous audio segment is less than a preset similarity, the current frame vector is generated as a new audio segment; if the similarity between the current frame vector and the previous audio segment is greater than or equal to the preset similarity, the current frame vector is merged into the previous audio segment.

[0123] ​If τ(t) is greater than or equal to a preset value, the current frame vector is generated into one or two new audio segments, thereby achieving upsampling. Specifically, when the similarity between the current frame vector and the previous audio segment is less than (1 - preset similarity), the current frame vector is generated into two identical new audio segments, which are achieved by copying the current frame vector; if the similarity between the current frame vector and the previous audio segment is greater than or equal to (1 - preset similarity), the current frame vector is generated into a new audio segment.

[0124] The distribution of the random threshold (i.e., the random sampling function) is determined by the percentile of the similarity between adjacent audio frequency bands. Since the percentile directly corresponds to the length of the resampled audio segment, the random threshold determines the numerical randomness, ensuring that the sampled audio frequency bands are of different lengths, which can eliminate the prosodic information of the original speaker.

[0125] In this embodiment, a random sampling function is obtained, and a random threshold for each frame vector in the latent feature vector is determined based on the random sampling function. Then, based on the magnitude of the random threshold and the preset value, each frame vector of the latent feature vector is upsampled or downsampled to obtain a resampled feature vector. This clarifies the specific process by which the resampling layer randomly samples the latent feature vector to obtain the resampled feature vector, ensuring that the sampled audio frequency bands have different lengths, which can eliminate the prosodic information of the original speaker.

[0126] In one embodiment, such as Figure 8 As shown, in step S43, the resampled feature vector is input into the decoding layer for decoding, and the target audio data is obtained by constructing speech from the decoding result based on the target domain identifier. Specifically, this includes the following steps:

[0127] S431: Input the resampled feature vector into the decoder of the decoding layer, and use the decoder to decode the resampled feature vector to obtain the decoded feature data;

[0128] S432: Input the target domain identifier and decoded feature data into the stream module of the decoding layer to construct the speech and obtain the target audio data.

[0129] After obtaining the resampling feature vector, the resampling feature vector is input into the decoder of the decoding layer, the resampling feature vector is decoded by the decoder to obtain decoded feature data, and then the target domain identifier and the decoded feature data are input into the flow module of the decoding layer to construct speech, to obtain target audio data. The flow module aims to establish a reversible modeling of the decoder to realize the streaming modeling of the speech and improve the audio conversion effect. The decoding layer of the embodiment aims to reconstruct the audio data according to the random sampling result of the resampling layer and the target domain identifier (such as the user identity ID of the expected audio quality and the ID of the expected emotional category), which can effectively convert the prosody style information given only the high-level global domain summary (such as the user identity ID of the expected audio quality and the ID of the expected emotional category), without directly inputting the expected audio quality and the expected emotional information, reducing the data processing amount, thereby improving the speech conversion efficiency.

[0130] In the embodiment, the resampling feature vector is input into the decoder of the decoding layer, the resampling feature vector is decoded by the decoder to obtain decoded feature data, and then the target domain identifier and the decoded feature data are input into the flow module of the decoding layer to construct speech, to obtain target audio data, which explicitly shows the specific process of inputting the resampling feature vector into the decoding layer for decoding, and constructing speech based on the target domain identifier to obtain the target audio data. The flow module realizes the streaming modeling of the speech and improves the audio conversion effect.

[0131] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0132] In an embodiment, an artificial intelligence-based audio generation device is provided, which corresponds one-to-one to the artificial intelligence-based audio generation method in the above embodiment. As shown in the figure, the artificial intelligence-based audio generation device includes A module, B module, C module and D module. The functions of each functional module are described in detail as follows: Figure 9

[0133] The first acquisition module 901 is configured to acquire the audio data to be converted, and acquire the target domain identifier corresponding to the expected audio quality and / or expected emotion;

[0134] The second acquisition module 902 is configured to acquire a speech conversion model, the speech conversion model being a deep learning model obtained by pre-training according to different audio data, and the speech conversion model including an encoding layer, a resampling layer and a decoding layer;

[0135] ​The feature conversion module 903 is configured to perform cepstrum feature conversion on the audio data to be converted based on a speech spectrum, to obtain cepstrum feature data of the audio data to be converted.

[0136] The speech conversion module 904 is configured to input the cepstrum feature data and the target domain identifier into a speech conversion model, to perform encoding on the cepstrum feature data at an encoding layer, to perform random sampling on the encoded vector at a resampling layer, and to perform decoding reconstruction on the sampled vector based on the target domain identifier at a decoding layer, to obtain target audio data.

[0137] Further, the feature conversion module 903 is specifically configured to:

[0138] perform Mel spectrum conversion on the audio data to be converted, to obtain a Mel spectrum of the audio data to be converted;

[0139] perform cepstrum analysis on the Mel spectrum of the audio data to be converted, to obtain a Mel frequency cepstrum coefficient of the audio data to be converted;

[0140] use the Mel frequency cepstrum coefficient of the audio data to be converted as the cepstrum feature data.

[0141] Further, the feature conversion module 903 is specifically configured to:

[0142] perform pre-emphasis processing on the audio data to be converted, to obtain pre-emphasis data;

[0143] perform frame processing on the pre-emphasis data, and perform windowing processing on each frame of the pre-emphasis data obtained by the frame processing, to obtain audio data to be processed;

[0144] perform Fourier transform on the audio data to be processed, to obtain a power spectrum of the audio data to be converted;

[0145] input the power spectrum of the audio data to be converted into a triangular filter based on a Mel scale to perform multiple filtering, to obtain a Mel spectrum of the audio data to be converted.

[0146] Further, the speech conversion module 904 is specifically configured to:

[0147] input the cepstrum feature data into the encoding layer to perform variational auto-encoding, to obtain a hidden state feature vector;

[0148] input the hidden state feature vector into the resampling layer to perform random sampling, to obtain a resampled feature vector;

[0149] input the resampled feature vector into the decoding layer to perform decoding, and perform speech construction on a decoding result based on the target domain identifier, to obtain the target audio data.

[0150] Further, the speech conversion module 904 is specifically configured to:

[0151] input the cepstrum feature data into an encoder of the encoding layer, and perform variational auto-encoding on the cepstrum feature data through the encoder to obtain an encoding feature vector of the cepstrum feature data;

[0152] perform normal distribution-based sampling on the encoding feature vector of the cepstrum feature data to obtain a hidden state feature vector.

[0153] Further, the speech conversion module 904 is specifically further configured to:

[0154] input the resampling feature vector into a decoder of the decoding layer, and perform decoding on the resampling feature vector through the decoder to obtain decoded feature data;

[0155] input the target domain identifier and the decoded feature data into a flow module of the decoding layer to perform speech construction to obtain target audio data.

[0156] Further, the speech conversion module 904 is specifically further configured to:

[0157] obtain a random sampling function, and determine a random threshold of each frame vector in the hidden state feature vector according to the random sampling function;

[0158] perform up-sampling or down-sampling on each frame vector in the hidden state feature vector according to the size of the random threshold and a preset value to obtain a resampling feature vector.

[0159] The specific limitations of the audio generation device based on artificial intelligence can be referred to the limitations of the audio generation method based on artificial intelligence in the foregoing, which will not be repeated here. Each module in the above audio generation device based on artificial intelligence can be realized by software, hardware and a combination thereof in whole or in part. The above each module can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0160] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 10As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is used to store the data generated and used by the artificial intelligence-based audio generation method. The network interface of the computer device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement an artificial intelligence-based audio generation method.

[0161] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the following steps:

[0162] Obtain the audio data to be converted, and obtain the target domain identifier corresponding to the expected sound quality and / or expected emotion;

[0163] Obtain a speech conversion model, the speech conversion model being a deep learning model obtained by pre-training according to different audio data, the speech conversion model comprising an encoding layer, a resampling layer and a decoding layer;

[0164] Perform cepstrum feature conversion based on the speech spectrum on the audio data to be converted to obtain cepstrum feature data of the audio data to be converted;

[0165] Input the cepstrum feature data and the target domain identifier into the speech conversion model to encode the cepstrum feature data at the encoding layer, randomly sample the encoded vector at the resampling layer, and decode and reconstruct the vector sampled at the decoding layer based on the target domain identifier to obtain target audio data.

[0166] In one embodiment, a computer readable storage medium is provided, having a computer program stored thereon, the computer program being executed by a processor to implement the following steps:

[0167] Obtain the audio data to be converted, and obtain the target domain identifier corresponding to the expected sound quality and / or expected emotion;

[0168] Obtain a speech conversion model, the speech conversion model being a deep learning model obtained by pre-training according to different audio data, the speech conversion model comprising an encoding layer, a resampling layer and a decoding layer;

[0169] Perform cepstrum feature conversion based on the speech spectrum on the audio data to be converted to obtain cepstrum feature data of the audio data to be converted;

[0170] The cepstrum feature data and the target domain identifier are input into a speech conversion model to encode the cepstrum feature data at an encoding layer, randomly sample the encoded vector at a resampling layer, and decode and reconstruct the sampled vector based on the target domain identifier at a decoding layer to obtain target audio data.

[0171] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0172] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0173] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An audio generation method based on artificial intelligence, characterized in that, include: Acquire the audio data to be converted, and acquire the target domain identifier corresponding to the desired sound quality and / or desired emotion, wherein the target domain identifier includes the source user's identity identifier for the desired sound quality, the emotion category symbol for the desired emotion, or the unique identifier corresponding to the voice information group; A speech conversion model is obtained, which is a deep learning model pre-trained based on different audio data. The speech conversion model includes an encoding layer, a resampling layer, and a decoding layer. The audio data to be converted is subjected to cepstral feature transformation based on the speech spectrum to obtain the cepstral feature data of the audio data to be converted; The cepstral feature data and the target domain identifier are input into the speech conversion model to encode the cepstral feature data in the coding layer, randomly sample the encoded vector in the resampling layer, and decode and reconstruct the sampled vector based on the target domain identifier in the decoding layer to obtain the target audio data. The process of encoding the cepstral feature data at the encoding layer, randomly sampling the encoded vector at the resampling layer, and decoding and reconstructing the sampled vector based on the target neighborhood identifier at the decoding layer to obtain target audio data includes: The cepstral feature data is input into the encoder of the coding layer, and the encoder performs variational autoencoding on the cepstral feature data to obtain the encoded feature vector of the cepstral feature data. The latent feature vector is obtained by sampling the encoded feature vector of the cepstral feature data based on a normal distribution; The hidden feature vector is input into the resampling layer for random sampling to obtain the resampled feature vector; The resampled feature vector is input into the decoding layer for decoding, and the target audio data is obtained by constructing speech based on the target domain identifier.

2. The audio generation method based on artificial intelligence as described in claim 1, characterized in that, The step of performing cepstral feature transformation on the audio data to be converted based on the speech spectrum to obtain cepstral feature data of the audio data to be converted includes: Perform Mel spectrum conversion on the audio data to be converted to obtain the Mel spectrum of the audio data to be converted; Perform cepstral analysis on the Mel frequency spectrum of the audio data to be converted to obtain the Mel frequency cepstral coefficients of the audio data to be converted; The Mel frequency cepstral coefficients of the audio data to be converted are used as the cepstral feature data.

3. The audio generation method based on artificial intelligence as described in claim 2, characterized in that, The step of performing Mel spectrum conversion on the audio data to be converted to obtain the Mel spectrum of the audio data to be converted includes: The audio data to be converted is pre-emphasized to obtain pre-emphasized data; The pre-emphasized data is divided into frames, and each frame of the pre-emphasized data is windowed to obtain the audio data to be processed. Perform a Fourier transform on the audio data to be processed to obtain the power spectrum of the audio data to be converted; The power spectrum of the audio data to be converted is input into a Mel-scale-based triangular filter for multiple filtering operations to obtain the Mel spectrum of the audio data to be converted.

4. The audio generation method based on artificial intelligence as described in claim 1, characterized in that, The step of inputting the resampled feature vector into the decoding layer for decoding, and constructing the target audio data by performing speech reconstruction on the decoding result based on the target domain identifier, includes: The resampled feature vector is input into the decoder of the decoding layer, and the decoder decodes the resampled feature vector to obtain the decoded feature data. The target domain identifier and the decoded feature data are input into the streaming module of the decoding layer to construct the speech, thereby obtaining the target audio data.

5. The audio generation method based on artificial intelligence as described in claim 1, characterized in that, The resampled feature vector is obtained in the following way: Obtain a random sampling function, and determine a random threshold for each frame vector in the latent feature vector based on the random sampling function; Based on the magnitude of the random threshold and the preset value, the vector of the latent feature vector in each frame is upsampled or downsampled to obtain the resampled feature vector.

6. An audio generation method based on artificial intelligence, characterized in that, include: The first acquisition module is used to acquire the audio data to be converted and to acquire the target domain identifier corresponding to the desired sound quality and / or desired emotion, wherein the target domain identifier includes the identity identifier of the source user of the desired sound quality, the emotion category symbol of the desired emotion, or the unique identifier corresponding to the voice information group. The second acquisition module is used to acquire a speech conversion model, which is a deep learning model pre-trained based on different audio data. The speech conversion model includes an encoding layer, a resampling layer, and a decoding layer. The feature conversion module is used to perform cepstral feature conversion on the audio data to be converted based on the speech spectrum to obtain the cepstral feature data of the audio data to be converted. The speech conversion module is used to input the cepstral feature data and the target domain identifier into the speech conversion model, to encode the cepstral feature data in the coding layer, to randomly sample the encoded vector in the resampling layer, and to decode and reconstruct the sampled vector based on the target domain identifier in the decoding layer to obtain the target audio data. The process of encoding the cepstral feature data at the encoding layer, randomly sampling the encoded vector at the resampling layer, and decoding and reconstructing the sampled vector based on the target neighborhood identifier at the decoding layer to obtain target audio data includes: The cepstral feature data is input into the encoder of the coding layer, and the encoder performs variational autoencoding on the cepstral feature data to obtain the encoded feature vector of the cepstral feature data. The latent feature vector is obtained by sampling the encoded feature vector of the cepstral feature data based on a normal distribution; The hidden feature vector is input into the resampling layer for random sampling to obtain the resampled feature vector; The resampled feature vector is input into the decoding layer for decoding, and the target audio data is obtained by constructing speech based on the target domain identifier.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the artificial intelligence-based audio generation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the artificial intelligence-based audio generation method as described in any one of claims 1 to 5.

Images