Standard voice generation method and device, medium and electronic equipment

Abstract

The invention relates to the technical field of voice processing, and provides a standard voice generation method and device, a medium and electronic equipment. The method comprises the steps of obtaining target accent feature sequence information through target voice; performing accent classification on the target accent feature sequence information to determine a target accent type; when the target accent type is a non-standard accent type, mapping the target accent feature sequence information to standard accent feature sequence information based on a mapping relationship between the target accent type and the standard accent type; and generating standard voice based on the standard accent feature sequence information. The non-standard target accent feature sequence information is effectively captured, the target accent feature sequence information is mapped to the standard accent feature sequence information through the mapping relation, and the recognition rate of the non-standard voice is improved.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology, and in particular to a method, device, medium and electronic device for generating standard speech. Background Art

[0002] In a globalized and diversified communication environment, the demand for cross-language and cross-cultural voice interaction is increasing.

[0003] However, speech recognition and conversion of non-standard accents has always been a challenge in the field of speech processing. Most existing speech recognition systems are optimized for standard accents (such as American English or British English), but the recognition accuracy of non-standard accents is low, making it difficult to meet diverse communication needs.

[0004] Therefore, the present application provides a method for generating a standard speech to solve the above technical problems. Summary of the invention

[0005] The purpose of this application is to provide a method, device, medium and electronic device for generating a standard voice, which can solve at least one of the above-mentioned technical problems. The specific solution is as follows: According to a specific implementation of the present application, in a first aspect, the present application provides a method for generating a standard speech, comprising: Get the target voice; Acquiring target accent feature sequence information based on the target speech; Performing accent classification on the target accent feature sequence information to determine the target accent type; When the target accent type is a non-standard accent type, mapping the target accent feature sequence information to the standard accent feature sequence information based on a mapping relationship between the target accent type and the standard accent type; Generate standard speech based on the standard accent feature sequence information.

[0006] Optionally, mapping the target accent feature sequence information to standard accent feature sequence information based on the mapping relationship between the target accent type and the standard accent type includes: The target accent feature sequence information is applied to a trained standard speech mapping model to obtain standard accent feature sequence information.

[0007] Optionally, the standard speech mapping model is generated based on a CycleGAN model.

[0008] Optionally, before applying the target accent feature sequence information to a trained standard speech mapping model to obtain the standard accent feature sequence information, the method further includes: Obtain multiple sample voices; Acquiring sample accent feature sequence information corresponding to the sample speech based on the multiple sample speech; An untrained standard speech mapping model is trained based on the sample accent feature sequence information of the multiple sample speech samples to obtain a trained standard speech mapping model, wherein the standard speech mapping model includes a generative adversarial loss function, specifically the following formula: ; in, L GAN (G, D, X, Y) represents the generative adversarial loss function, G Table Generator Network, D represents the discriminator network, X The sample space representing the sequence information of non-standard accent features, x ~ X express x is from X Sampling from the distribution of Y The sample space representing the standard accent feature sequence information, y ~ Y express y is from Y The expected operator E is obtained by sampling from the distribution of, which means averaging the samples in the sample space. E y～Y [logD(y)] Expressing All samples obtained from ,calculate The average value of The standard speech mapping model also includes a cycle consistency loss function, which is specifically the following formula: ; in, L cycle (G, F) represents the cycle consistency loss function, F express G The inverse mapping generator is used to convert the standard accent feature sequence information y Convert back to non-standard accent feature sequence information ( ),and G Together they form a loop conversion structure, ||.||1 means L1 The norm (sum of absolute values) is used to measure the degree of difference between the transformed eigenvector and the original eigenvector. Indicates that the original non-standard accent feature sequence information Through the generator Converted into standard accent feature sequence information, and then passed through the generator GAfter conversion back to non-standard accent feature sequence information, it is different from the original non-standard accent feature sequence information Between Norm distance, that is, the size of the difference between them, Indicates that the original standard accent feature sequence information y Through the generator Converted into non-standard accent feature sequence information, and then passed through the generator G After converting back to the standard accent feature sequence information, it is different from the original standard accent feature sequence information Between Norm distance, that is, the size of the difference between them.

[0009] Optionally, before applying the target accent feature sequence information to a trained standard speech mapping model to obtain the standard accent feature sequence information, the method further includes: Random noise information is added to the target accent feature sequence information.

[0010] Optionally, performing accent classification on the target accent feature sequence information to determine the target accent type includes: The target accent feature sequence information is classified based on the trained Transformer model to determine the target accent type.

[0011] Optionally, the target accent feature sequence information includes target accent spectral feature information and / or target accent prosodic feature information.

[0012] According to a specific implementation of the present application, in a second aspect, the present application provides a device for generating a standard speech, including: A target acquisition unit, used for acquiring target speech; A feature acquisition unit, configured to acquire target accent feature sequence information based on the target speech; a determination unit, configured to perform accent classification on the target accent feature sequence information to determine a target accent type; a mapping unit, configured to map the target accent feature sequence information to the standard accent feature sequence information based on a mapping relationship between the target accent type and the standard accent type when the target accent type is a non-standard accent type; A generating unit is used to generate standard speech based on the standard accent feature sequence information.

[0013] Optionally, mapping the target accent feature sequence information to standard accent feature sequence information based on the mapping relationship between the target accent type and the standard accent type includes: The target accent feature sequence information is applied to a trained standard speech mapping model to obtain standard accent feature sequence information.

[0014] Optionally, the standard speech mapping model is generated based on a CycleGAN model.

[0015] Optionally, the device further comprises: A sample acquisition unit, used for applying the target accent feature sequence information to a trained standard speech mapping model, and acquiring a plurality of sample speech before acquiring the standard accent feature sequence information; A sample feature acquisition unit, configured to acquire sample accent feature sequence information corresponding to the sample speech based on the plurality of sample speech; A training unit is used to train an untrained standard speech mapping model based on the sample accent feature sequence information of the multiple sample speech samples to obtain a trained standard speech mapping model, wherein the standard speech mapping model includes generating an adversarial loss function, specifically the following formula: ; in, L GAN (G, D, X, Y) represents the generative adversarial loss function, G represents the generator network, D represents the discriminator network, X The sample space representing the sequence information of non-standard accent features, x ~ X express x is from X Sampling from the distribution of Y The sample space representing the standard accent feature sequence information, y ~ Y express y is from Y The expected operator E is obtained by sampling from the distribution of, which means averaging the samples in the sample space. E y～Y [logD(y)] Expressing All samples obtained from ,calculate The average value of The standard speech mapping model also includes a cycle consistency loss function, which is specifically the following formula: ; in, L cycle (G, F) represents the cycle consistency loss function, F express G The inverse mapping generator is used to convert the standard accent feature sequence information yConvert back to non-standard accent feature sequence information ( ),and G Together they form a loop conversion structure, ||.||1 means L1 The norm (sum of absolute values) is used to measure the degree of difference between the transformed eigenvector and the original eigenvector. Indicates that the original non-standard accent feature sequence information Through the generator Converted into standard accent feature sequence information, and then passed through the generator G After conversion back to non-standard accent feature sequence information, it is different from the original non-standard accent feature sequence information Between Norm distance, that is, the size of the difference between them, Indicates that the original standard accent feature sequence information y Through the generator Converted into non-standard accent feature sequence information, and then passed through the generator G After converting back to the standard accent feature sequence information, it is different from the original standard accent feature sequence information Between Norm distance, that is, the size of the difference between them.

[0016] Optionally, before applying the target accent feature sequence information to a trained standard speech mapping model to obtain the standard accent feature sequence information, the method further includes: Random noise information is added to the target accent feature sequence information.

[0017] Optionally, before training an untrained standard speech mapping model based on the sample accent feature sequence information of the plurality of sample speech samples to obtain a trained standard speech mapping model, the method further includes: Random noise information is added to the sample accent feature sequence information of the plurality of sample speech.

[0018] Optionally, performing accent classification on the target accent feature sequence information to determine the target accent type includes: The target accent feature sequence information is classified based on the trained Transformer model to determine the target accent type.

[0019] Optionally, the target accent feature sequence information includes target accent spectral feature information and / or target accent prosodic feature information.

[0020] According to the specific implementation of the present application, in a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method for generating a standard speech as described in any of the above items is implemented.

[0021] According to the specific implementation of the present application, in a fourth aspect, the present application provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method for generating a standard speech as described in any of the above items.

[0022] Compared with the prior art, the above solution of the embodiment of the present application has at least the following beneficial effects: The present application provides a method, device, medium and electronic device for generating a standard speech. The present application obtains target accent feature sequence information through target speech; performs accent classification on the target accent feature sequence information to determine the target accent type; when the target accent type is a non-standard accent type, maps the target accent feature sequence information to the standard accent feature sequence information based on the mapping relationship between the target accent type and the standard accent type; and generates a standard speech based on the standard accent feature sequence information. The non-standard target accent feature sequence information is effectively captured, and the target accent feature sequence information is mapped to the standard accent feature sequence information through a mapping relationship, thereby improving the recognition rate of non-standard speech and realizing real-time speech conversion. The system is suitable for real-time communication and interactive scenarios, adapts to different language environments, improves the adaptability and coverage in non-standard accent areas, and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart showing a method for generating a standard voice according to an embodiment of the present application is shown; Figure 2 A unit block diagram of a device for generating standard speech according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0025] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0026] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0027] It should be understood that although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0028] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0029] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0030] It should be particularly noted that any symbols and / or numbers in the specification that are not marked in the accompanying drawings are not drawing marks.

[0031] The optional embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0032] The embodiment provided in this application is an embodiment of a method for generating standard speech.

[0033] Combine the following Figure 1 The embodiments of the present application are described in detail.

[0034] Step S101, obtaining target speech.

[0035] The target speech may be the original speech, or the speech obtained after the original speech is standardized and noise suppressed, but the embodiments of the present application are not limited thereto.

[0036] Step S102: acquiring target accent feature sequence information based on the target speech.

[0037] The target accent feature sequence information is obtained by performing a series of preprocessing and feature extraction steps on the target speech. First, the audio signal of the target speech is framed, and the continuous audio signal is divided into shorter time frames, such as 20-40 milliseconds per frame. There can be a certain overlap between adjacent frames (such as 50% overlap). Then, feature extraction is performed on each frame of the audio signal. These features can characterize the acoustic characteristics of the audio signal in each frame, such as timbre, formant and other information. The features extracted from each frame are arranged in chronological order to form the target accent feature sequence information.

[0038] In some specific embodiments, the target accent feature sequence information includes target accent spectral feature information and / or target accent prosodic feature information.

[0039] This specific embodiment uses short-time Fourier transform (STFT) to extract the target spoken audio spectrum feature information in the target speech, which specifically includes the following formula: ; in, Indicates the target spoken audio spectrum feature information, represents the target voice, is the window function, is the window size.

[0040] Indicates at time and frequency It is the result of short-time Fourier transform of the target speech, reflecting the energy distribution of the signal at different times and frequencies.

[0041] The time variable representing the target speech on the time axis is used to locate and analyze the target speech on the time axis. In the short-time Fourier transform, the target speech in different time periods is processed by moving the window function on the time axis. The time position of the current processing is determined.

[0042] Represents frequency variables, which are used to represent frequency components in the spectrum. By analyzing the spectrum values ​​under the target speech, we can get the frequency characteristics of the target speech, such as which frequency components are stronger and which are weaker.

[0043] Represents a discrete time index value used to traverse the target speech It is an integer variable, from arrive , represents the position of the sample point within the current window function; The signal sequence representing the target speech is a discrete sample representation of the target speech that needs to be spectrally analyzed. For example, a series of values ​​obtained after sampling the target speech. Indicates the sequence number of each sample.

[0044] Represents a sequence of window functions, which is used to perform windowing on the target speech. The role of the window function is to intercept and weight the target speech on the time axis, reduce problems such as spectrum leakage, and improve the accuracy of spectrum analysis. Common window functions include Hanning window and Hamming window, and different window functions have different characteristics.

[0045] Indicates the window size, that is, the window function The length of the window. It determines the number of samples of the target speech considered each time the short-time Fourier transform analysis is performed. A larger window size can provide better frequency resolution, but the time resolution will be reduced; a smaller window size is the opposite, with better time resolution but poorer frequency resolution. In practical applications, it is necessary to select an appropriate window size based on the characteristics of the target speech and the analysis requirements.

[0046] represents an imaginary unit, satisfying , in the index term is used to represent complex operations, so that the short-time Fourier transform can convert the time domain signal to the frequency domain while retaining the phase information of the target speech.

[0047] Represents pi, a mathematical constant. It is used in exponential terms to determine the relationship between frequency and time points and is one of the basic parameters in Fourier transform; represents a natural constant, approximately equal to , as the base in the exponential term, is used to construct the complex exponential form and realize the conversion of the signal from the time domain to the frequency domain. It is actually a rotation factor, which is based on the frequency , Time Index and window size To determine the position and phase of each sample in the frequency domain.

[0048] This specific embodiment performs windowing processing on the target speech, and uses the principle of discrete Fourier transform (DFT) (implemented by exponential terms here) to calculate the target spoken audio spectrum feature information.

[0049] This specific embodiment extracts the target spoken audio spectral feature information in the target speech through the Mel-frequency cepstral coefficients and pitch, which specifically includes the following formula: ; in, Indicates Mel frequency cepstral coefficients. MFCC is a set of coefficients used to characterize the characteristics of speech signals. Used to distinguish different coefficients. Usually the first few (such as or ) coefficients to represent speech signals, because they can effectively capture important information such as speech timbre and resonance peaks, and serve as key feature vectors in tasks such as speech recognition and speech synthesis.

[0050] Represents the index value of the Mel filter, ranging from arrive , used to traverse all Mel filters. Different Mel filters corresponding to different center frequencies are used to perform weighted analysis on the spectrum of the speech signal in different frequency ranges, thereby obtaining feature information related to different frequencies and jointly forming the MFCC vector.

[0051] Indicates the frequency index value, which is used to traverse the frequency components of the target speech after spectrum analysis (such as STFT). arrive , which indicates the different frequency points considered when calculating the spectrum. k By processing the corresponding spectral components, complete spectral information can be obtained, and then the frequency-related speech features can be calculated.

[0052] It means the first spectral components. It is a complex number that represents the target speech at a specific time frame and frequency. The spectrum value at includes the amplitude and phase values ​​of the target speech. The amplitude value reflects the energy of the frequency component in the target speech.

[0053] Represents the spectral components The square operation of the modulus obtains the energy of the spectral component, which reflects the intensity distribution of the target speech at different frequencies, helps to distinguish different speech phonemes, formants and other features, and provides an important basis for subsequent Mel filtering and MFCC calculation.

[0054] Indicates The Mel filter is The Mel filter bank is a set of filters designed according to the Mel frequency scale, which aims to simulate the nonlinear perception characteristics of the human auditory system to frequency. Corresponding to filters with different center frequencies, Indicates The filter at frequency The weight value of the input spectrum energy is and By multiplying and summing, we can get the energy distribution after Mel filtering, which makes it more consistent with the frequency characteristics of human auditory perception.

[0055] Indicates logarithmic operation, which is usually the natural logarithm (i.e. ) or the common logarithm ( Logarithmic operations have two main functions in MFCC calculation: one is to compress the dynamic range of data, because the energy difference of different frequency components in speech signals may be large, and logarithmic operations can make small energy changes more obvious and avoid overflow and underflow problems in numerical calculations; the other is to convert multiplication operations into addition operations, which is more convenient in subsequent calculations and analysis. For example, when calculating operations such as vector distance, calculations in the logarithmic domain can simplify the calculation process and improve calculation efficiency.

[0056] represents the window size used in spectrum analysis (such as STFT), N include or .

[0057] Indicates the number of filters in the Mel filter bank. It determines the dimension of the final MFCC vector (optionally, plus a coefficient representing energy, such as order coefficient). include ~ . M The larger the value, the more detailed the speech features on the Mel frequency scale can be captured, but it will also increase the amount of calculation and the dimension of the feature vector. The value needs to balance the feature expression ability and computational complexity. For example, a smaller value may be chosen when resources are limited. In tasks that require high feature accuracy, a larger value may be selected. value.

[0058] Step S103: performing accent classification on the target accent feature sequence information to determine the target accent type.

[0059] In some specific embodiments, performing accent classification on the target accent feature sequence information to determine the target accent type includes: Step S103a: performing accent classification on the target accent feature sequence information based on the trained Transformer model to determine the target accent type.

[0060] This specific embodiment uses a trained Transformer model to perform accent classification on the target accent feature sequence information. The Transformer model does not have an intrinsic sequential information processing mechanism like a recurrent neural network, but instead uses position encoding and self-attention mechanisms to capture long-range dependencies in the target accent feature sequence information. Position encoding provides the Transformer model with position-related clues, which helps improve the Transformer model's ability to understand and process features at different positions in the sequence.

[0061] Specifically, the position encoding includes the following formula: ; ; in, Represents the encoding information added to each position (corresponding to each frame in the target accent feature sequence information). The purpose of position encoding is to enable the Transformer model to perceive the position order of elements in the input sequence.

[0062] Represents the position index value in the target accent feature sequence information. For the audio feature sequence, It can correspond to the sequence number of the audio frame, starting from 0 and increasing in sequence. For example, represents the first frame in the sequence, Indicates the second frame, and so on. ,Position encoding can assign different encoding values ​​to audio frames at different positions, thereby distinguishing their positions in the sequence.

[0063] Represents the dimension index value used to calculate the position encoding. In the Transformer model, the position encoding is usually a vector with the same dimension as the hidden layer of the model. Used to traverse this dimension. For example, if the model hidden layer dimension is ,So The value range is from arrive The positional encoding is constructed by using sine and cosine functions in different dimensions. Determines the dimension in which the encoded value at a particular position is calculated.

[0064] Represents the hidden layer dimension of the Transformer model. It determines the complexity and capacity of the model's internal representation and also affects how the positional encoding is calculated. The frequency of the positional encoding is based on To determine, different This will cause the position encoding to change at different frequencies, thus providing the model with position information of different resolutions. It enables the Transformer model to learn more complex feature representations, but it also increases the computational cost and the number of parameters.

[0065] and Represents sine and cosine functions, which are used to generate periodic changes in position encoding. By using sine and cosine functions, position encoding can show periodic changes in different dimensions. This periodicity helps the model capture the relative position relationship in the target accent feature sequence information. For example, for adjacent positions, their position encodings will have similar change patterns in some dimensions, so that the model can perceive the continuity of the position; while there will be different changes in other dimensions to provide sufficient discrimination.

[0066] It is a constant value used to control the frequency of position encoding. In the position encoding formula, it is used as the base of the exponent and is and Together they determine the frequencies of the sine and cosine functions. A larger base will cause the positional encoding to change slowly in the low-frequency part and quickly in the high-frequency part, thereby providing positional information of different frequencies in different dimensions, which helps the model distinguish the importance and feature differences of different positions. When the Transformer model is used to identify the target accent type of the target accent feature sequence information, the target accent feature sequence information provides an acoustic feature representation of the audio signal, while the positional encoding supplements the Transformer model with positional information. The combination of the two enables the model to better process audio sequence data and improve the accuracy of accent recognition.

[0067] Step S104: when the target accent type is a non-standard accent type, mapping the target accent feature sequence information to standard accent feature sequence information based on a mapping relationship between the target accent type and the standard accent type.

[0068] In some specific embodiments, mapping the target accent feature sequence information to the standard accent feature sequence information based on the mapping relationship between the target accent type and the standard accent type includes: Step S104a: applying the target accent feature sequence information to a trained standard speech mapping model to obtain standard accent feature sequence information.

[0069] In some specific embodiments, before applying the target accent feature sequence information to a trained standard speech mapping model to obtain the standard accent feature sequence information, the method further includes: adding random noise information to the target accent feature sequence information.

[0070] In this specific embodiment, random noise information is added to the target accent feature sequence information to simulate the uncertainty and variability in the target speech, making the generated standard speech more natural and diverse. The random noise information is a random vector value that matches the dimension of the feature vector, for example, the random noise information includes a normally distributed random number or a uniformly distributed random number.

[0071] In some specific embodiments, the standard speech mapping model is generated based on a CycleGAN model.

[0072] The output of the standard speech mapping model is the converted standard accent feature sequence information corresponding to the input non-standard target accent feature sequence information. The standard accent feature sequence information is similar in form to the input non-standard target accent feature sequence information. For example, if the target accent feature sequence information includes target accent spectral feature information and / or target accent prosodic feature information, then the standard accent feature sequence information includes standard accent spectral feature information and / or standard accent prosodic feature information. Specifically, the standard accent spectral feature information output by the standard speech mapping model may be closer to the characteristics of standard speech in terms of resonance peak structure, frequency distribution, etc., and the standard accent prosodic feature information output by the standard speech mapping model will also be adjusted to the prosodic pattern of standard speech. For example, the pitch change pattern, duration distribution, etc. are more in line with the prosodic pattern of standard speech.

[0073] This specific embodiment builds a conversion model from non-standard accent feature sequence information to standard accent feature sequence information based on the CycleGAN model, and realizes the mapping relationship between non-standard accent feature sequence information and standard accent feature sequence information.

[0074] In some specific embodiments, in order to train a standard speech mapping model, before applying the target accent feature sequence information to the trained standard speech mapping model to obtain the standard accent feature sequence information, the method further includes: Step S100-1, obtaining a plurality of sample voices.

[0075] The sample speech may be the original speech, or the speech obtained after the original speech is standardized and noise suppressed, but the embodiments of the present application are not limited thereto.

[0076] Step S100 - 3 : acquiring sample accent feature sequence information corresponding to the sample speech based on the multiple sample speech.

[0077] The method for obtaining the sample accent feature sequence information is the same as the method for obtaining the target accent feature sequence information, which will not be described in detail here. Please refer to the method for obtaining the target accent feature sequence information.

[0078] Step S100 - 5 , training an untrained standard speech mapping model based on the sample accent feature sequence information of the plurality of sample speech samples to obtain a trained standard speech mapping model.

[0079] The standard speech mapping model includes a discriminator and a generator.

[0080] The output of the discriminator can reflect the similarity or difference between the generated standard accent feature sequence information and the real standard accent feature sequence information. By analyzing this information, the training strategy of the standard speech mapping model can be adjusted to improve the generation quality.

[0081] The intermediate state of the generator can help researchers understand the internal mechanism of the standard speech mapping model during the feature conversion process and discover possible problems, such as whether the feature mapping is reasonable, whether there is information loss, etc. The standard speech mapping model processes the input non-standard target accent feature sequence information and outputs the converted standard accent feature sequence information, thereby achieving the conversion task of non-standard target accent feature sequence information, providing technical support for improving cross-accent voice communication, meeting diverse user needs, and improving the adaptability of voice interaction.

[0082] The standard speech mapping model includes generating an adversarial loss function, which is specifically the following formula: ; in, L GAN (G, D, X, Y) represents the generative adversarial loss function, G represents the generator network, D represents the discriminator network, X The sample space representing the sequence information of non-standard accent features, x ~ X express x is from X Sampling from the distribution of Y The sample space representing the standard accent feature sequence information, y ~ Y express yis from Y The expected operator E is obtained by sampling from the distribution of, which means averaging the samples in the sample space. E y～Y [logD(y)] Expressing All samples obtained from ,calculate The average value of The standard speech mapping model also includes a cycle consistency loss function, which is specifically the following formula: ; in, L cycle (G, F) represents the cycle consistency loss function, F express G The inverse mapping generator is used to convert the standard accent feature sequence information y Convert back to non-standard accent feature sequence information ( ),and G Together they form a loop conversion structure, ||.||1 means L1 The norm (sum of absolute values) is used to measure the degree of difference between the transformed eigenvector and the original eigenvector. Indicates that the original non-standard accent feature sequence information Through the generator Converted into standard accent feature sequence information, and then passed through the generator G After conversion back to non-standard accent feature sequence information, it is different from the original non-standard accent feature sequence information Between Norm distance, that is, the size of the difference between them, Indicates that the original standard accent feature sequence information y Through the generator Converted into non-standard accent feature sequence information, and then passed through the generator G After converting back to the standard accent feature sequence information, it is different from the original standard accent feature sequence information Between Norm distance, that is, the size of the difference between them.

[0083] In some specific embodiments, the step of training an untrained standard speech mapping model based on the sample accent feature sequence information of the plurality of sample speech samples to obtain a trained standard speech mapping model further includes: Step S100 - 4 , adding random noise information to the sample accent feature sequence information of the plurality of sample speech.

[0084] In this specific embodiment, random noise information is added to the sample accent feature sequence information to simulate the uncertainty and variability in the sample speech, making the generated standard speech more natural and diverse. The random noise information is a random vector value that matches the dimension of the feature vector, for example, the random noise information includes a normally distributed random number or a uniformly distributed random number.

[0085] Step S105: Generate standard speech based on the standard accent feature sequence information.

[0086] The WaveNet model can be used to generate standard speech from standard accent feature sequence information.

[0087] The embodiment of the present application obtains target accent feature sequence information through target speech; performs accent classification on the target accent feature sequence information to determine the target accent type; when the target accent type is a non-standard accent type, maps the target accent feature sequence information to the standard accent feature sequence information based on the mapping relationship between the target accent type and the standard accent type; and generates standard speech based on the standard accent feature sequence information. The non-standard target accent feature sequence information is effectively captured, and the target accent feature sequence information is mapped to the standard accent feature sequence information through a mapping relationship, thereby improving the recognition rate of non-standard speech and realizing real-time speech conversion. The system is suitable for real-time communication and interactive scenarios, adapts to different language environments, improves adaptability and coverage in non-standard accent areas, and improves user experience.

[0088] The present application also provides a device embodiment that is based on the above embodiment, which is used to implement the method steps described in the above embodiment. The explanation based on the same name meaning is the same as the above embodiment, and has the same technical effect as the above embodiment, which will not be repeated here.

[0089] like Figure 2 As shown, the present application provides a device 200 for generating a standard speech, comprising: A target acquisition unit 201, used to acquire a target speech; A feature acquisition unit 202, configured to acquire target accent feature sequence information based on the target speech; A determination unit 203, configured to perform accent classification on the target accent feature sequence information to determine a target accent type; A mapping unit 204, configured to map the target accent feature sequence information to standard accent feature sequence information based on a mapping relationship between the target accent type and the standard accent type when the target accent type is a non-standard accent type; The generating unit 205 is configured to generate standard speech based on the standard accent feature sequence information.

[0090] Optionally, mapping the target accent feature sequence information to standard accent feature sequence information based on the mapping relationship between the target accent type and the standard accent type includes: The target accent feature sequence information is applied to a trained standard speech mapping model to obtain standard accent feature sequence information.

[0091] Optionally, the standard speech mapping model is generated based on a CycleGAN model.

[0092] Optionally, the device further comprises: A sample acquisition unit, used for applying the target accent feature sequence information to a trained standard speech mapping model, and acquiring a plurality of sample speech before acquiring the standard accent feature sequence information; A sample feature acquisition unit, configured to acquire sample accent feature sequence information corresponding to the sample speech based on the plurality of sample speech; A training unit is used to train an untrained standard speech mapping model based on the sample accent feature sequence information of the multiple sample speech samples to obtain a trained standard speech mapping model, wherein the standard speech mapping model includes generating an adversarial loss function, specifically the following formula: ; in, L GAN (G, D, X, Y) represents the generative adversarial loss function, G represents the generator network, D represents the discriminator network, X The sample space representing the sequence information of non-standard accent features, x ~ X express x is from X Sampling from the distribution of Y The sample space representing the standard accent feature sequence information, y ~ Y express y is from Y The expected operator E is obtained by sampling from the distribution of, which means averaging the samples in the sample space. E y～Y [logD(y)] Expressing All samples obtained from ,calculate The average value of The standard speech mapping model also includes a cycle consistency loss function, which is specifically the following formula: ; in, L cycle (G, F)represents the cycle consistency loss function, F express G The inverse mapping generator is used to convert the standard accent feature sequence information y Convert back to non-standard accent feature sequence information ( ),and G Together they form a loop conversion structure, ||.||1 means L1 The norm (sum of absolute values) is used to measure the degree of difference between the transformed eigenvector and the original eigenvector. Indicates that the original non-standard accent feature sequence information Through the generator Converted into standard accent feature sequence information, and then passed through the generator G After conversion back to non-standard accent feature sequence information, it is different from the original non-standard accent feature sequence information Between Norm distance, that is, the size of the difference between them, Indicates that the original standard accent feature sequence information y Through the generator Converted into non-standard accent feature sequence information, and then passed through the generator G After converting back to the standard accent feature sequence information, it is different from the original standard accent feature sequence information Between Norm distance, that is, the size of the difference between them.

[0093] Optionally, before applying the target accent feature sequence information to a trained standard speech mapping model to obtain the standard accent feature sequence information, the method further includes: Random noise information is added to the target accent feature sequence information.

[0094] Optionally, before training an untrained standard speech mapping model based on the sample accent feature sequence information of the plurality of sample speech samples to obtain a trained standard speech mapping model, the method further includes: Random noise information is added to the sample accent feature sequence information of the plurality of sample speech.

[0095] Optionally, performing accent classification on the target accent feature sequence information to determine the target accent type includes: The target accent feature sequence information is classified based on the trained Transformer model to determine the target accent type.

[0096] Optionally, the target accent feature sequence information includes target accent spectral feature information and / or target accent prosodic feature information.

[0097] The embodiment of the present application obtains target accent feature sequence information through target speech; performs accent classification on the target accent feature sequence information to determine the target accent type; when the target accent type is a non-standard accent type, maps the target accent feature sequence information to the standard accent feature sequence information based on the mapping relationship between the target accent type and the standard accent type; and generates standard speech based on the standard accent feature sequence information. The non-standard target accent feature sequence information is effectively captured, and the target accent feature sequence information is mapped to the standard accent feature sequence information through a mapping relationship, thereby improving the recognition rate of non-standard speech and realizing real-time speech conversion. The system is suitable for real-time communication and interactive scenarios, adapts to different language environments, improves adaptability and coverage in non-standard accent areas, and improves user experience.

[0098] Example 3 This embodiment provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method steps described in the above embodiment.

[0099] Example 4 An embodiment of the present application provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the method steps described in the above embodiment.

[0100] Finally, it should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0101] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a standard voice, characterized in that: include: Get the target voice; Acquiring target accent feature sequence information based on the target speech; Performing accent classification on the target accent feature sequence information to determine the target accent type; When the target accent type is a non-standard accent type, mapping the target accent feature sequence information to the standard accent feature sequence information based on a mapping relationship between the target accent type and the standard accent type; Generate standard speech based on the standard accent feature sequence information.

2. The method according to claim 1, characterized in that The mapping of the target accent feature sequence information to the standard accent feature sequence information based on the mapping relationship between the target accent type and the standard accent type includes: The target accent feature sequence information is applied to a trained standard speech mapping model to obtain standard accent feature sequence information.

3. The method according to claim 2, characterized in that The standard speech mapping model is generated based on the CycleGAN model.

4. The method according to claim 3, characterized in that Before applying the target accent feature sequence information to the trained standard speech mapping model to obtain the standard accent feature sequence information, the method further includes: Obtain multiple sample voices; Acquiring sample accent feature sequence information corresponding to the sample speech based on the multiple sample speech; An untrained standard speech mapping model is trained based on the sample accent feature sequence information of the multiple sample speech samples to obtain a trained standard speech mapping model, wherein the standard speech mapping model includes a generative adversarial loss function, specifically the following formula: ； in, L GAN (G, D, X, Y) represents the generative adversarial loss function, G represents the generator network, D represents the discriminator network, X The sample space representing the sequence information of non-standard accent features, x~X express x is from X Sampling from the distribution of Y The sample space representing the standard accent feature sequence information, y~Y express y is from Y The expected operator E is obtained by sampling from the distribution of, which means averaging the samples in the sample space. E y～Y [logD(y)] Expressing All samples obtained from ,calculate The average value of The standard speech mapping model also includes a cycle consistency loss function, which is specifically the following formula: ； in, L cycle (G, F) represents the cycle consistency loss function, F express G The inverse mapping generator is used to convert the standard accent feature sequence information y Convert back to non-standard accent feature sequence information ( ),and G Together they form a loop conversion structure, ||.||1 means L1 The norm (sum of absolute values) is used to measure the degree of difference between the transformed eigenvector and the original eigenvector. Indicates that the original non-standard accent feature sequence information Through the generator Converted into standard accent feature sequence information, and then passed through the generator G After conversion back to non-standard accent feature sequence information, it is different from the original non-standard accent feature sequence information Between Norm distance, that is, the size of the difference between them, Indicates that the original standard accent feature sequence information y Through the generator Converted into non-standard accent feature sequence information, and then passed through the generator G After converting back to the standard accent feature sequence information, it is different from the original standard accent feature sequence information Between Norm distance, that is, the size of the difference between them.

5. The method according to claim 2, characterized in that: Before applying the target accent feature sequence information to the trained standard speech mapping model to obtain the standard accent feature sequence information, the method further includes: Random noise information is added to the target accent feature sequence information.

6. The method according to claim 1, characterized in that The performing accent classification on the target accent feature sequence information to determine the target accent type includes: The target accent feature sequence information is classified based on the trained Transformer model to determine the target accent type.

7. The method according to claim 1, characterized in that The target accent feature sequence information includes target accent spectral feature information and / or target accent prosodic feature information.

8. A device for generating a standard voice, characterized in that: include: A target acquisition unit, used for acquiring target speech; A target feature acquisition unit, configured to acquire target accent feature sequence information based on the target speech; a determination unit, configured to perform accent classification on the target accent feature sequence information to determine a target accent type; a mapping unit, configured to map the target accent feature sequence information to the standard accent feature sequence information based on a mapping relationship between the target accent type and the standard accent type when the target accent type is a non-standard accent type; A generating unit is used to generate standard speech based on the standard accent feature sequence information.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in any one of claims 1 to 7.

Images