Voice conversion method, device, equipment, storage medium and program product

Abstract

The application relates to a speech conversion method, device, equipment, storage medium and program product. The method comprises the following steps: obtaining original semantic features corresponding to to-be-converted speech data; adjusting the speech length corresponding to the original semantic features through alignment information to obtain target semantic features; the alignment information is generated according to the mapping relationship between a plurality of original semantic features and corresponding standard semantic features; and target speech data converted from the to-be-converted speech data is determined through the target semantic features. The method can accelerate the speech conversion speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a voice conversion method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] With the development of computer technology, voice conversion technology has emerged.

[0003] Traditional voice conversion technology first extracts text information from a voice waveform, and then uses an autoregressive method to output converted voice frames frame by frame based on the voice text information, thereby achieving simultaneous optimization of recognition and synthesis, and enhancing the conversion success rate in voice conversion technology.

[0004] However, the voice frame output after frame-by-frame conversion is time-consuming for each voice frame conversion process, which limits its real-time use in live scenarios. SUMMARY

[0005] Therefore, it is necessary to provide a voice conversion method, device, equipment, storage medium and program product capable of accelerating the conversion speed of voice frames to solve the above technical problems.

[0006] In a first aspect, the present application provides a voice conversion method, comprising: obtaining original semantic features corresponding to to-be-converted voice data; adjusting the voice length of the original semantic features corresponding to the original semantic features through alignment information to obtain target semantic features; the alignment information is generated according to the mapping relationship between a plurality of original semantic features and corresponding standard semantic features; determining the target voice data converted from the to-be-converted voice data through the target semantic features.

[0007] In one embodiment, the voice conversion method further comprises: obtaining probability information between the original semantic features and the corresponding standard semantic features according to the mapping relationship between the plurality of original semantic features and the corresponding standard semantic features; and obtaining the alignment information according to the probability information between the original semantic features and the corresponding standard semantic features.

[0008] In one embodiment, the obtaining of the original semantic features corresponding to the to-be-converted voice data comprises: performing conversion processing on the to-be-converted voice data to obtain first spectral data of the to-be-converted voice data; and inputting the first spectral data into a pre-set feature extraction network to obtain the original semantic features.

[0009] In one of the embodiments, the construction process of the feature extraction network comprises: obtaining spectral data of a plurality of voice data; inputting the spectral data of the plurality of voice data into an initial prediction module in an initial feature extraction network to obtain predicted semantic features, and inputting the predicted semantic features and training semantic features into an initial first module to obtain a first phoneme sequence, and inputting the predicted semantic features into an initial second module to obtain a second phoneme sequence; obtaining loss information according to the first phoneme sequence, the second phoneme sequence and the training semantic features; if the loss information satisfies a preset iteration condition, the feature extraction network is obtained.

[0010] In one of the embodiments, the conversion processing of the voice data to be converted to obtain first spectral data of the voice data to be converted comprises: conversion processing of the voice data to be converted to obtain initial spectral data of the voice data to be converted; pre-processing of the initial spectral data to obtain the first spectral data; the pre-processing at least comprises voice endpoint detection.

[0011] In one of the embodiments, the target voice data converted from the voice data to be converted is determined by the target semantic features, comprising: conversion of the target semantic features to obtain second spectral data; generation of the target voice data converted from the voice data to be converted according to the second spectral data.

[0012] In one of the embodiments, the voice conversion method further comprises: determining speech rate adjustment information according to the alignment information and a preset coefficient; adjusting the speech rate of the voice data to be converted according to the speech rate adjustment information.

[0013] In a second aspect, the present application further provides a voice conversion device, comprising: a feature acquisition module configured to acquire original semantic features corresponding to voice data to be converted; a target feature acquisition module configured to adjust a voice length of the original semantic features according to alignment information to obtain target semantic features; the alignment information is generated according to a mapping relationship between a plurality of original semantic features and corresponding standard semantic features; and a target data conversion module configured to determine target voice data converted from the voice data to be converted by the target semantic features.

[0014] In a third aspect, the present application further provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0015] In a fourth aspect, the present application further provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0016] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of the method described above.

[0017] The voice conversion method, device, equipment, storage medium and program product described above can adjust the voice length corresponding to the original semantic feature through the alignment information, i.e., the original semantic feature can be converted into the corresponding target semantic feature at one time through the alignment information, thereby accelerating the conversion speed of the voice. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 An application environment diagram of the voice conversion method in an embodiment;

[0019] Figure 2 A flowchart of the voice conversion method in an embodiment;

[0020] Figure 3 A schematic diagram of a voice waveform in an embodiment;

[0021] Figure 4 A schematic diagram of a mel-frequency spectrum in an embodiment;

[0022] Figure 5 A schematic diagram of the construction of a feature extraction network in an embodiment;

[0023] Figure 6 A flowchart of the voice conversion method in an embodiment;

[0024] Figure 7 A structural schematic diagram corresponding to part of the implementation steps of the voice conversion method in an embodiment;

[0025] Figure 8 A structural block diagram of the voice conversion device in an embodiment;

[0026] Figure 9 An internal structure diagram of the computer equipment in an embodiment. DETAILED DESCRIPTION

[0027] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0028] The voice conversion method provided in the application can be applied to a computer device, which can be a terminal, a server, or a device cluster composed of a terminal and a server, such as various personal computers, notebook computers, tablet computers, wearable devices, and the like, but the type of the computer device is not limited in the embodiments of the application. As shown in Figure 1 An internal structure diagram of a computer device is provided, Figure 1 The processor in the computer device is configured to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database is configured to store related data of a feature selection process. The network interface is configured to communicate with other devices outside through a network connection. The computer program is executed by the processor to implement a voice conversion method.

[0029] In one embodiment, as shown in Figure 2 A voice conversion method is provided, which is applied to a terminal 102 in Figure 1 for example, and includes the following steps:

[0030] In step 202, original semantic features corresponding to the voice data to be converted are obtained.

[0031] The voice data to be converted is original voice data that is intended to be converted. The original semantic features are the smallest units of voice that can distinguish the meaning in the voice data to be converted.

[0032] Specifically, the terminal obtains each smallest unit of voice corresponding to the voice data to be converted.

[0033] Optionally, the voice data to be converted is in the form of voice waveform to be converted, and the semantic features are phonemes included in the voice data to be converted.

[0034] In the specific implementation process, the terminal obtains the voice waveform to be converted corresponding to the voice data to be converted, and obtains the phonemes included in the voice waveform to be converted, that is, the terminal obtains the phoneme sequence constituting the voice waveform to be converted.

[0035] In step 204, the length of the voice corresponding to the original semantic features is adjusted through alignment information to obtain target semantic features. The alignment information is generated according to the mapping relationship between the original semantic features and the corresponding standard semantic features.

[0036] The alignment information refers to the mapping relationship between the original semantic features and the standard semantic features.

[0037] Specifically, the terminal obtains a phoneme sequence corresponding to the speech waveform to be converted, obtains alignment information from phonemes in the phoneme sequence corresponding to the speech waveform to be converted and phonemes in a phoneme sequence corresponding to a target speech waveform to be converted, and performs a copying or merging operation on the phonemes in the phoneme sequence corresponding to the speech waveform to be converted according to the alignment information to form a new phoneme sequence, which is a sequence of target semantic features. The copying or merging operation on the phonemes in the phoneme sequence corresponding to the speech waveform to be converted according to the alignment information adjusts the length of the original semantic feature.

[0038] In a specific implementation process, taking a phoneme sequence corresponding to the speech waveform to be converted obtained by the terminal as X = {X1, X2, X3} for example, alignment information D = {2, 1, 3} is obtained from phonemes X1, X2, X3 in the phoneme sequence corresponding to the speech waveform to be converted and phonemes in a phoneme sequence corresponding to a target speech waveform to be converted, and then the phonemes in the phoneme sequence corresponding to the speech waveform to be converted are adjusted according to the alignment information to form a new phoneme sequence, which is target semantic features Y = {X1, X1, X2, X3, X3, X3}. The meaning of Y = {X1, X1, X2, X3, X3, X3} is that X1 phonemes in the target semantic features correspond to 2 frames of the target speech waveform, while the pronunciation duration of X1 phonemes in the speech waveform to be converted is 1 frame, and the corresponding alignment information is 2.

[0039] In step 206, the target speech data after conversion of the speech data to be converted is determined according to the target semantic features.

[0040] Specifically, the terminal converts the sequence of target semantic features into corresponding target speech data.

[0041] In the speech conversion method, the length of the original semantic features is adjusted according to the alignment information, that is, the original semantic features can be converted into corresponding target semantic features at one time according to the alignment information, thereby accelerating the conversion speed of the speech.

[0042] In one embodiment, the speech conversion method further includes: obtaining probability information between the original semantic features and the corresponding standard semantic features according to a mapping relationship between the original semantic features and the corresponding standard semantic features; and obtaining the alignment information according to the probability information between the original semantic features and the corresponding standard semantic features.

[0043] The standard semantic features are standard semantic features of the target speech data to be converted.

[0044] Specifically, assuming that original semantic features X = {X1, X2, X3, …, XT} need to be converted into target semantic features Y = {Y1, Y2, Y3, …, YT’}, since different voice tones and speeds will make the voice duration of the same content different, i.e., T and T’ are different, in order to realize the alignment of T and T’, the probability information between the original semantic features and the corresponding standard semantic features needs to be obtained, and the corresponding alignment information can be obtained after vitebi decoding of the probability information. The manner of obtaining the probability information is as follows:

[0045] Assuming that a{t,s} = p(Y{1:t}|H{1:s}) is the probability of the frame number 1-s frame of a certain original semantic feature in the voice data to be converted and the frame number 1-t frame of the corresponding target semantic feature. The corresponding relationship between the frame number 1-s frame of a certain original semantic feature and the frame number 1-t frame of the corresponding target semantic feature is obtained according to the original semantic features and the corresponding standard semantic features.

[0046] The iteration formula is a{t,s} = (a{t-1,s} + a{t,s-1})*p(Y{t}|H{s})

[0047] L{Y,H} = -log(p(Y[1:T’]|H{1:T*})) = -log a{T’,T*}

[0048] In the voice conversion method, the probability information between the original semantic features and the corresponding standard semantic features is obtained according to the mapping relationship between the original semantic features and the corresponding standard semantic features, so that the acquisition of the alignment information is quantifiable and convenient to realize.

[0049] In one embodiment, the voice conversion method further includes: determining speech speed adjustment information according to the alignment information and a preset coefficient; and adjusting the speech speed of the voice data to be converted according to the speech speed adjustment information.

[0050] Specifically, according to the above embodiment, the terminal obtains the alignment information, multiplies the alignment information D by a preset coefficient p to obtain speech speed adjustment information p*D, and adjusts the speech speed of the voice data to be converted according to the speech speed adjustment information.

[0051] In the voice conversion method, the speech speed adjustment information is determined by the alignment information and the preset coefficient, and the speech speed adjustment of the voice corresponding to the voice data to be converted is realized.

[0052] In one embodiment, the original semantic features corresponding to the voice data to be converted are obtained by: performing conversion processing on the voice data to be converted to obtain first spectral data of the voice data to be converted; and inputting the first spectral data into a preset feature extraction network to obtain the original semantic features.

[0053] Optionally, the first spectral data is a Mel spectrum. This embodiment does not limit the type of the first spectral data of the speech data to be converted; it can be any spectral data corresponding to the speech data to be converted.

[0054] Specifically, taking the voice data to be converted as an example... Figure 3 Taking the speech waveform to be converted as an example, the terminal converts the speech waveform to be converted into the corresponding waveform through pre-emphasis, framing, windowing, fast Fourier transform, and Mel-spectral transform. Figure 4 The speech Mel spectrum shown is used as the first spectral data. The terminal inputs the first spectral data into a preset feature extraction network to obtain the phoneme sequence that constitutes the first spectral data, where the phonemes in the phoneme sequence are the original semantic features.

[0055] In the aforementioned speech conversion method, the speech data to be converted is processed to obtain the first spectral data of the speech data to be converted. This makes the speech data to be converted appear as spectral data, thus reducing the dimensionality of the speech data to be converted. This preserves as much information as possible that the human ear needs to understand the original speech, reducing the cost (resource consumption) in the training and use of the speech conversion model. The terminal inputs the first spectral data into a preset feature extraction network to obtain the original semantic features, realizing the recognition of the minimum original semantic features of the first spectral data, thus preparing for subsequent speech conversion.

[0056] In one embodiment, the construction process of the feature extraction network includes: acquiring spectral data of multiple speech data; inputting the spectral data of the multiple speech data into the initial prediction module of the initial feature extraction network to obtain predicted semantic features; inputting the predicted semantic features and training semantic features into the initial first module to obtain a first phoneme sequence; and inputting the predicted semantic features into the initial second module to obtain a second phoneme sequence. Loss information is obtained based on the first phoneme sequence, the second phoneme sequence, and the training semantic features; if the loss information satisfies a preset iteration condition, the feature extraction network is obtained.

[0057] Optionally, the spectral data is Mel spectrum data, meaning the terminal acquires the Mel spectra corresponding to multiple speech data as training samples. The construction process of the feature extraction network is as follows: Figure 5 As shown, the terminal inputs the training samples into the initial prediction module. Figure 5 The encoder module outputs predicted semantic features, which are the smallest distinguishable speech units in the Mel spectrum of the sample. The initial prediction module consists of a two-layer convolutional neural network and a multi-layer multi-head attention network. The terminal inputs the predicted semantic features into the multi-layer occlusion multi-head attention network of the initial first module and the initial second module. The initial first module consists of... Figure 5The right side is composed of an Embedding layer, a multi-layer occlusion multi-head attention network module, and a softmax module. The initial second module is composed of a CTC module. The initial first module outputs a first phoneme sequence after learning. The initial second module outputs a second phoneme sequence, that is, a Phoneme sequence of CTC. The input of the Embedding layer of the initial first module is a training semantic feature. The training semantic feature is the input of the encoder module, that is, the pre-trained training semantic feature of the training sample. The initial first module learns according to the predicted semantic feature and the training speech feature to obtain the first phoneme sequence. The CTC algorithm of the initial second module learns according to the predicted semantic feature and the training semantic feature to obtain the second phoneme sequence. The terminal obtains first loss information through the first phoneme sequence and the training semantic feature, and obtains second loss information through the second phoneme sequence and the training semantic feature. The two loss information jointly guide the learning of the feature extraction network. When the value of the above loss information meets the preset iteration condition, the feature extraction network is obtained. The iteration condition is a local optimal solution of the loss information, and the embodiment does not limit how to obtain the local optimal solution. Alternatively, the phonemes in the phoneme sequence and the minimum speech unit in the embodiment are Phonemes.

[0058] In the construction process of the feature extraction network in the above speech conversion method, the two modules are used to train the semantic features extracted from the speech data. The two modules jointly guide the learning of the entire network, promote each other, accelerate the network learning convergence, and improve the accuracy of semantic feature extraction. The speech conversion method of the present application cooperates with the speech conversion method to improve the speed of semantic feature extraction.

[0059] In one embodiment, the conversion processing is performed on the to-be-converted speech data to obtain first spectral data of the to-be-converted speech data, including: performing conversion processing on the to-be-converted speech data to obtain initial spectral data of the to-be-converted speech data; and pre-processing the initial spectral data to obtain the first spectral data; the pre-processing at least includes speech endpoint detection.

[0060] Specifically, taking the to-be-converted speech data as an example, the terminal converts the to-be-converted speech waveform into corresponding speech mel spectrum as initial spectral data through pre-emphasis, framing, windowing, fast Fourier transform, and mel spectrum transform. The terminal detects each starting endpoint of the initial spectral data that needs to be changed by speech endpoint detection, cuts the initial mel spectrum graph according to each starting endpoint, and takes each initial mel spectrum after cutting as first spectral data to filter non-real speech such as silence noise in the initial mel spectrum. The above non-real speech is a sound emitted by a non-human.

[0061] In the above speech conversion method, speech endpoint detection is used to preprocess the speech data to be converted, so as to filter out non-real speech in the speech data to be converted, thereby reducing the amount of computation required for subsequent extraction and alignment of original semantic features.

[0062] In one embodiment, determining the target speech data after conversion of the speech data to be converted by the target semantic features includes: converting the target semantic features to obtain second spectral data; and generating the target speech data after conversion of the speech data to be converted based on the second spectral data.

[0063] Specifically, the terminal converts the target semantic features into second spectral data using a speech synthesis decoder. Optionally, the second spectral data is of the Mel spectrum type. Optionally, the speech synthesis decoder model structure employs a multi-layer multi-head attention network. The terminal then uses HiFi-gan (a generative adversarial network for efficient and high-fidelity speech synthesis) to generate a speech waveform, i.e., the target speech data, from the second spectral data of the Mel spectrum type.

[0064] In the above speech conversion method, the terminal converts the target semantic features to obtain the second spectrum data, and then generates the target speech data after the speech data to be converted. The above speech conversion method is efficient and of good quality.

[0065] In one embodiment, such as Figure 6 As shown, the format of the speech data to be converted is waveform. The terminal acquires the speech waveform to be converted as the speech data to be converted, and inputs the above speech waveform to be converted into the speech signal processing module. The terminal performs pre-emphasis, framing, windowing, fast Fourier transform, and Mel-spectrum transform steps through the signal processing module to convert the speech waveform to be converted into a two-dimensional original Mel-spectrum diagram with acoustic features. The two-dimensional original Mel-spectrum diagram with acoustic features corresponding to the above speech waveform to be converted is used as the output of the speech signal processing module, i.e., the initial spectrum data.

[0066] The terminal inputs the initial spectral data output from the speech signal processing module to the speech endpoint detection module. The speech endpoint detection module detects the starting endpoints in the initial spectral data that require voice modification, segments the initial Mel spectrogram based on these starting endpoints, and uses the segmented initial Mel spectrogram as the output of the speech endpoint detection module, i.e., the first spectral data. The terminal then uses the speech endpoint detection module to eliminate the influence of silence in the initial spectral data, extracting the valid speech with human voice.

[0067] The terminal inputs the first spectral data output by the voice endpoint detection module into the semantic information feature coding module, and outputs a feature hidden vector representing semantic information of the voice as original semantic features. The terminal extracts semantic information of the voice segment through the semantic information feature coding module, and eliminates timbre information of the original voice. It should be noted that the semantic information feature coding module is an ASR recognition model based on a ctc-attention framework for extracting phoneme information. Only the encoding part of the ASR recognition model is needed during test running, and the encoding part of the ASR recognition model is the semantic information feature coding module.

[0068] The terminal inputs the original semantic features output by the semantic information feature coding module into the voice feature changer, adjusts the timbre duration through the voice length changer of the voice feature changer, generates corresponding target semantic features through the original semantic features through alignment information, and takes the target semantic features as the output of the voice feature changer. Optionally, the modeling information in the voice feature changer also includes pitch, speed, and volume information, so as to change the pitch, speed, and volume of the initial semantic features. Among them, the voice length changer trains and tests the alignment information through aligloss (semantic segmentation model). Figure 7 For example, the semantic information feature coding module adjusts the volume, pitch, and length through the volume adjuster, pitch predictor, and length predictor.

[0069] The terminal inputs the target semantic features output by the voice feature changer into the voice synthesis decoding module, and outputs a mel-spectrogram corresponding to the target semantic features as the second spectral data. Optionally, the network structure of the voice synthesis decoding module can adopt a transformer (deep self-attention transformation network, an autoregressive model) or a BiLSTM (bidirectional long short-term memory model) network structure.

[0070] The terminal inputs the second spectral data into the voice vocoder to convert the second spectral data into voice waveform data, and outputs the voice waveform data as target voice data.

[0071] It should be understood that although each step in the flowchart involved in the above-described embodiments is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowchart involved in the above-described embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential but can be executed in rotation or alternation with at least some of the other steps or the steps or stages in the other steps.

[0072] Based on the same inventive concept, the embodiments of the present application also provide a voice conversion device for implementing the voice conversion method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more voice conversion device embodiments provided below can refer to the limitations of the voice conversion method described above, which will not be repeated here.

[0073] In one embodiment, as shown in Figure 8 A voice conversion device is provided, comprising: a feature acquisition module 100, a target feature acquisition module 200, and a target data conversion module 300, wherein:

[0074] The feature acquisition module 100 is configured to acquire original semantic features corresponding to the voice data to be converted.

[0075] The target feature acquisition module 200 is configured to adjust the voice length of the original semantic features by using alignment information to obtain target semantic features; the alignment information is generated according to the mapping relationship between the plurality of original semantic features and the corresponding standard semantic features.

[0076] The target data conversion module 300 is configured to determine the target voice data converted from the voice data to be converted by using the target semantic features.

[0077] In one embodiment, the target feature acquisition module 200 comprises: acquiring probability information between the original semantic features and the corresponding standard semantic features according to the mapping relationship between the plurality of original semantic features and the corresponding standard semantic features; and acquiring the alignment information according to the probability information between the original semantic features and the corresponding standard semantic features.

[0078] In one embodiment, the feature acquisition module 100 comprises: performing conversion processing on the voice data to be converted to obtain first spectral data of the voice data to be converted; and inputting the first spectral data into a preset feature extraction network to obtain the original semantic features.

[0079] In one embodiment, the construction process of the feature extraction network comprises: a spectrum data acquisition module configured to acquire spectrum data of a plurality of voice data; a phoneme sequence acquisition module configured to input the spectrum data of the plurality of voice data into an initial prediction module in an initial feature extraction network to obtain predicted semantic features, and input the predicted semantic features and training semantic features into an initial first module to obtain a first phoneme sequence, and input the predicted semantic features into an initial second module to obtain a second phoneme sequence; and a network determination module configured to obtain loss information according to the first phoneme sequence, the second phoneme sequence and the training semantic features, and obtain the feature extraction network if the loss information meets a preset iteration condition.

[0080] In one embodiment, the feature acquisition module 100 comprises: converting the voice data to be converted to obtain initial spectrum data of the voice data to be converted; and pre-processing the initial spectrum data to obtain first spectrum data; the pre-processing at least comprises voice endpoint detection.

[0081] In one embodiment, the target data conversion module 300 comprises: converting the target semantic features to obtain second spectrum data; and generating target voice data converted from the voice data to be converted according to the second spectrum data.

[0082] In one embodiment, the voice conversion apparatus further comprises: a determination adjustment information module configured to determine speech rate adjustment information according to the alignment information and a preset coefficient; and a speech rate adjustment module configured to adjust the speech rate of the voice data to be converted according to the speech rate adjustment information.

[0083] The above voice conversion apparatus can be implemented by software, hardware and a combination thereof. The above modules can be embedded in or independent of a processor in a computer device in a hardware form, or stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to the above modules.

[0084] In one embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram of the computer device can be as shown in Figure 9The computer device shown in the figure includes a processor, a memory, a communication interface, and a display screen connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved through WIFI, mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program is executed by the processor to implement a voice conversion method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen.

[0085] Those skilled in the art can understand that Figure 9 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0086] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps: obtaining original semantic features corresponding to the to-be-converted voice data; adjusting the voice length corresponding to the original semantic features through alignment information to obtain target semantic features; the alignment information is generated according to the mapping relationship between the original semantic features and the corresponding standard semantic features; determining the target voice data converted from the to-be-converted voice data through the target semantic features.

[0087] In one embodiment, the processor executing the computer program further implements the following steps: obtaining probability information between the original semantic features and the corresponding standard semantic features according to the mapping relationship between the original semantic features and the corresponding standard semantic features; and obtaining the alignment information according to the probability information between the original semantic features and the corresponding standard semantic features.

[0088] In one embodiment, the processor executing the computer program further implements the following steps: performing conversion processing on the to-be-converted voice data to obtain first spectral data of the to-be-converted voice data; and inputting the first spectral data into a preset feature extraction network to obtain the original semantic features.

[0089] In one embodiment, the construction process of the feature extraction network comprises: obtaining spectral data of a plurality of speech data; inputting the spectral data of the plurality of speech data into an initial prediction module in an initial feature extraction network to obtain predicted semantic features, and inputting the predicted semantic features and training semantic features into an initial first module to obtain a first phoneme sequence, and inputting the predicted semantic features into an initial second module to obtain a second phoneme sequence; obtaining loss information according to the first phoneme sequence, the second phoneme sequence and the training semantic features; if the loss information meets a preset iteration condition, obtaining the feature extraction network.

[0090] In one embodiment, the processor further implements the following steps when executing the computer program: performing conversion processing on the speech data to be converted to obtain initial spectral data of the speech data to be converted; performing preprocessing on the initial spectral data to obtain first spectral data; the preprocessing at least includes speech endpoint detection.

[0091] In one embodiment, the processor further implements the following steps when executing the computer program: converting the target semantic features to obtain second spectral data; generating target speech data converted from the speech data to be converted according to the second spectral data.

[0092] In one embodiment, the processor further implements the following steps when executing the computer program: determining speech speed adjustment information according to the alignment information and a preset coefficient; adjusting the speech speed of the speech data to be converted according to the speech speed adjustment information.

[0093] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the following steps: obtaining original semantic features corresponding to speech data to be converted; adjusting the speech length of the original semantic features by using alignment information to obtain target semantic features; the alignment information is generated according to the mapping relationship between a plurality of original semantic features and corresponding standard semantic features; determining target speech data converted from the speech data to be converted by using the target semantic features.

[0094] In one embodiment, the computer program is executed by the processor to further implement the following steps: obtaining probability information between the original semantic features and the corresponding standard semantic features according to the mapping relationship between the plurality of original semantic features and the corresponding standard semantic features; obtaining the alignment information according to the probability information between the original semantic features and the corresponding standard semantic features.

[0095] In one embodiment, the computer program is executed by the processor to further implement the following steps: performing conversion processing on the speech data to be converted to obtain first spectral data of the speech data to be converted; inputting the first spectral data into a preset feature extraction network to obtain the original semantic features.

[0096] In one embodiment, the construction process of the feature extraction network comprises: obtaining spectral data of a plurality of voice data; inputting the spectral data of the plurality of voice data into an initial prediction module in an initial feature extraction network to obtain predicted semantic features, and inputting the predicted semantic features and training semantic features into an initial first module to obtain a first phoneme sequence, and inputting the predicted semantic features into an initial second module to obtain a second phoneme sequence; obtaining loss information according to the first phoneme sequence, the second phoneme sequence and the training semantic features; if the loss information meets a preset iteration condition, obtaining the feature extraction network.

[0097] In one embodiment, the computer program is further implemented by the processor to perform the following steps: converting the voice data to be converted to obtain initial spectral data of the voice data to be converted; preprocessing the initial spectral data to obtain first spectral data; the preprocessing at least includes voice endpoint detection.

[0098] In one embodiment, the computer program is further implemented by the processor to perform the following steps: converting the target semantic features to obtain second spectral data; generating target voice data converted from the voice data to be converted according to the second spectral data.

[0099] In one embodiment, the computer program is further implemented by the processor to perform the following steps: determining speech rate adjustment information according to the alignment information and a preset coefficient; adjusting the speech rate of the voice data to be converted according to the speech rate adjustment information.

[0100] In one embodiment, a computer program product is provided, comprising a computer program, which is implemented by the processor to perform the following steps: obtaining original semantic features corresponding to the voice data to be converted; adjusting the voice length of the original semantic features through alignment information to obtain target semantic features; the alignment information is generated according to the mapping relationship between a plurality of original semantic features and corresponding standard semantic features; determining target voice data converted from the voice data to be converted through the target semantic features.

[0101] In one embodiment, the computer program is further implemented by the processor to perform the following steps: obtaining probability information between the original semantic features and the corresponding standard semantic features according to the mapping relationship between the plurality of original semantic features and the corresponding standard semantic features; obtaining the alignment information according to the probability information between the original semantic features and the corresponding standard semantic features.

[0102] In one embodiment, the computer program is further implemented by the processor to perform the following steps: converting the voice data to be converted to obtain first spectral data of the voice data to be converted; inputting the first spectral data into a preset feature extraction network to obtain the original semantic features.

[0103] In one embodiment, the construction process of the feature extraction network comprises: obtaining spectral data of a plurality of voice data; inputting the spectral data of the plurality of voice data into an initial prediction module in an initial feature extraction network to obtain predicted semantic features, and inputting the predicted semantic features and training semantic features into an initial first module to obtain a first phoneme sequence, and inputting the predicted semantic features into an initial second module to obtain a second phoneme sequence; obtaining loss information according to the first phoneme sequence, the second phoneme sequence and the training semantic features; if the loss information meets a preset iteration condition, obtaining the feature extraction network.

[0104] In one embodiment, the computer program is further implemented when executed by the processor to perform the following steps: converting the voice data to be converted to obtain initial spectral data of the voice data to be converted; preprocessing the initial spectral data to obtain first spectral data; the preprocessing at least comprises voice endpoint detection.

[0105] In one embodiment, the computer program is further implemented when executed by the processor to perform the following steps: converting the target semantic features to obtain second spectral data; generating target voice data converted from the voice data to be converted according to the second spectral data.

[0106] In one embodiment, the computer program is further implemented when executed by the processor to perform the following steps: determining speech rate adjustment information according to the alignment information and a preset coefficient; adjusting the speech rate of the voice data to be converted according to the speech rate adjustment information.

[0107] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0108] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0109] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0110] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A voice conversion method characterized by, The method comprises: obtaining original semantic features corresponding to the to-be-converted voice data; wherein the original semantic features are the smallest phonetic units capable of distinguishing meanings in the voice included in the to-be-converted voice data, and the original semantic features include phonemes in a phoneme sequence; adjusting the voice length of the original semantic features by alignment information to obtain target semantic features; the alignment information is generated according to the mapping relationship between the plurality of original semantic features and corresponding standard semantic features; determining target voice data converted from the to-be-converted voice data by the target semantic features; The voice conversion method further comprises: obtaining probability information between the original semantic features and corresponding standard semantic features according to the mapping relationship between the plurality of original semantic features and corresponding standard semantic features; obtaining the alignment information according to the probability information between the original semantic features and corresponding standard semantic features.

2. The method of claim 1, wherein, The obtaining of the original semantic features corresponding to the to-be-converted voice data comprises: performing conversion processing on the to-be-converted voice data to obtain first spectral data of the to-be-converted voice data; inputting the first spectral data into a preset feature extraction network to obtain the original semantic features.

3. The method of claim 2, wherein, The construction process of the feature extraction network comprises: obtaining spectral data of a plurality of voice data; inputting the spectral data of the plurality of voice data into an initial prediction module in an initial feature extraction network to obtain predicted semantic features, inputting the predicted semantic features and training semantic features into an initial first module to obtain a first phoneme sequence, and inputting the predicted semantic features into an initial second module to obtain a second phoneme sequence; obtaining loss information according to the first phoneme sequence, the second phoneme sequence and the training semantic features; if the loss information satisfies a preset iteration condition, the feature extraction network is obtained.

4. The method of claim 2, wherein, The conversion processing on the to-be-converted voice data to obtain the first spectral data of the to-be-converted voice data comprises: performing conversion processing on the to-be-converted voice data to obtain initial spectral data of the to-be-converted voice data; preprocessing the initial spectral data to obtain the first spectral data; the preprocessing at least comprises voice endpoint detection.

5. The method of claim 1, wherein, The determination of the target voice data converted from the to-be-converted voice data by the target semantic features comprises: performing conversion on the target semantic features to obtain second spectral data; generating target voice data converted from the to-be-converted voice data according to the second spectral data.

6. The method of claim 1, wherein, The voice conversion method further comprises: determining speech rate adjustment information according to the alignment information and a preset coefficient; adjusting the speech rate of the to-be-converted voice data according to the speech rate adjustment information.

7. An acoustic transducer device, characterized by The device comprises: a feature acquisition module configured to obtain original semantic features corresponding to to-be-converted voice data; wherein the original semantic features are the smallest phonetic units capable of distinguishing meanings in the voice included in the to-be-converted voice data, and the original semantic features include phonemes in a phoneme sequence; The target feature acquisition module is configured to adjust the speech length corresponding to the original semantic feature by using alignment information to obtain a target semantic feature, wherein the alignment information is generated according to a mapping relationship between a plurality of original semantic features and corresponding standard semantic features; The target data conversion module is configured to determine target speech data converted from the to-be-converted speech data by using the target semantic feature. The target feature acquisition module is further configured to acquire probability information between the original semantic feature and the corresponding standard semantic feature according to a mapping relationship between the plurality of original semantic features and the corresponding standard semantic features, and acquire the alignment information according to the probability information between the original semantic feature and the corresponding standard semantic feature.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images