Voice conversion method, device, computer equipment, storage medium and program product

Abstract

The present application relates to a speech conversion method, apparatus, computer device, storage medium, and computer program product. The method comprises: obtaining a speech to be converted, extracting speech features and semantic information from the speech to be converted, obtaining target speech features corresponding to the speech features of the speech to be converted based on a preset speech feature correspondence, and then generating a target converted speech based on the target speech features and the semantic information of the speech to be converted. This method allows the semantic information of the speech to be converted to be retained while only the speech features are replaced, achieving conversion from the speech to the target converted speech. This method eliminates the need for speech conversion based on a neural network speech conversion model, reduces the computational complexity of the conversion process, and thereby shortens conversion time and improves efficiency.

Description

Technical Field

[0001] The present application relates to the field of voice data processing technology, and in particular to a voice conversion method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0002] With the development of computer technology, speech conversion is increasingly being used in our daily lives. In practice, speech conversion includes various types, including speech conversion between different languages, speech conversion between different accents, speech conversion between different timbres, and so on.

[0003] Traditionally, voice conversion between different timbres has been achieved using a voice conversion model trained using a large amount of source and target speaker speech data. Specifically, the source speaker's speech data is input into the voice conversion model, which then performs a series of feature extraction and transformations on the source speaker's speech data to output voice data converted to the target speaker's timbre, achieving timbre conversion from the source speaker to the target speaker.

[0004] However, the above-mentioned speech conversion model has a complex structure, the conversion process requires a lot of calculation, the conversion is time-consuming and inefficient. Summary of the Invention

[0005] Based on this, it is necessary to provide a speech conversion method, apparatus, computer device, computer-readable storage medium and computer program product to address the above technical issues.

[0006] In a first aspect, the present application provides a speech conversion method, comprising:

[0007] Acquire the speech to be converted, and extract the speech features and semantic information of the speech to be converted;

[0008] According to a preset speech feature correspondence relationship, a target speech feature corresponding to the speech feature of the speech to be converted is obtained; wherein the speech feature correspondence relationship includes at least one group of speech feature correspondence relationships;

[0009] The target converted speech is generated according to the target speech features and the semantic information of the speech to be converted.

[0010] In one embodiment, if the speech feature correspondence includes a set of speech feature correspondences, and the speech feature correspondences represent a correspondence between a first speech feature and a second speech feature;

[0011] According to the preset speech feature correspondence relationship, the target speech feature corresponding to the speech feature of the speech to be converted is obtained, including:

[0012] Calculating feature similarities between the speech feature of the speech to be converted and the first speech feature and the second speech feature;

[0013] The speech feature with the minimum feature similarity between the first speech feature and the second speech feature is determined as the target speech feature.

[0014] In one embodiment, if the speech feature correspondence includes multiple groups of speech feature correspondences, the process of constructing the speech feature correspondence includes:

[0015] Obtaining multiple source voices and target voices corresponding to each source voice;

[0016] Extracting a first speech feature of each source speech and a second speech feature of each target speech;

[0017] The corresponding first speech features and second speech features are associated to obtain multiple groups of speech feature correspondences.

[0018] In one embodiment, obtaining target speech features corresponding to speech features of the speech to be converted according to a preset speech feature correspondence relationship includes:

[0019] Determining a target correspondence relationship among the multiple groups of speech feature correspondence relationships based on a feature similarity between a speech feature of the speech to be converted and a first speech feature in the multiple groups of speech feature correspondence relationships;

[0020] The target speech features are determined according to the target correspondence.

[0021] In one embodiment, determining a target correspondence relationship from a plurality of groups of speech feature correspondence relationships based on a feature similarity between a speech feature of the speech to be converted and a first speech feature in the feature conversion relationship includes:

[0022] Obtaining feature similarity between the speech feature of the speech to be converted and the first speech feature in each group of speech feature correspondences;

[0023] Determine the first speech feature whose feature similarity is greater than a similarity threshold as the target first speech feature;

[0024] A target correspondence relationship in which the first voice feature matches the target first voice feature is determined among the multiple groups of voice feature correspondence relationships.

[0025] In one embodiment, extracting speech features and semantic information of the speech to be converted includes:

[0026] Input the speech to be converted into a feature extraction model to obtain speech features of the speech to be converted;

[0027] The speech to be converted is input into the semantic extraction model to obtain the semantic information of the speech to be converted.

[0028] In a second aspect, the present application further provides a speech conversion device, comprising:

[0029] A speech processing module is used to obtain the speech to be converted and extract the speech features and semantic information of the speech to be converted;

[0030] A target determination module is configured to obtain target speech features corresponding to speech features of the speech to be converted based on a preset speech feature correspondence relationship; wherein the speech feature correspondence relationship includes at least one group of speech feature correspondence relationships;

[0031] The speech generation module is used to generate the target converted speech according to the target speech features and the semantic information of the speech to be converted.

[0032] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0033] Acquire the speech to be converted, and extract the speech features and semantic information of the speech to be converted;

[0034] According to a preset speech feature correspondence relationship, a target speech feature corresponding to the speech feature of the speech to be converted is obtained; wherein the speech feature correspondence relationship includes at least one group of speech feature correspondence relationships;

[0035] The target converted speech is generated according to the target speech features and the semantic information of the speech to be converted.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0037] Acquire the speech to be converted, and extract the speech features and semantic information of the speech to be converted;

[0038] According to a preset speech feature correspondence relationship, a target speech feature corresponding to the speech feature of the speech to be converted is obtained; wherein the speech feature correspondence relationship includes at least one group of speech feature correspondence relationships;

[0039] The target converted speech is generated according to the target speech features and the semantic information of the speech to be converted.

[0040] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0041] Acquire the speech to be converted, and extract the speech features and semantic information of the speech to be converted;

[0042] According to a preset speech feature correspondence relationship, a target speech feature corresponding to the speech feature of the speech to be converted is obtained; wherein the speech feature correspondence relationship includes at least one group of speech feature correspondence relationships;

[0043] The target converted speech is generated according to the target speech features and the semantic information of the speech to be converted.

[0044] The aforementioned speech conversion method, apparatus, computer device, storage medium, and computer program product obtain the speech to be converted, extract the speech features and semantic information of the speech to be converted, and then, based on a preset speech feature correspondence, obtain target speech features corresponding to the speech features of the speech to be converted. Furthermore, the target converted speech is generated based on the target speech features and the semantic information of the speech to be converted. This method achieves conversion from the speech to the target converted speech by replacing only the speech features while retaining the semantic information of the speech to be converted. This method eliminates the need for speech conversion based on a neural network speech conversion model, reduces the computational complexity of the conversion process, and thus shortens conversion time and improves efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 1 is a flow chart of a method for voice conversion according to an embodiment;

[0046] Figure 2 A schematic diagram of a process for determining target speech features in one embodiment;

[0047] Figure 3 A schematic diagram of a process for constructing a speech feature correspondence relationship in one embodiment;

[0048] Figure 4 A schematic structural diagram of a conversion device in one embodiment;

[0049] Figure 5 A schematic diagram of a flow chart for determining target speech features in another embodiment;

[0050] Figure 6 A schematic diagram of a process for determining a target correspondence relationship in one embodiment;

[0051] Figure 7 A schematic diagram of a process for obtaining speech features and semantic information in one embodiment;

[0052] Figure 8 A schematic diagram of a flow chart for determining target speech features in another embodiment;

[0053] Figure 9 is a structural block diagram of a speech conversion device in one embodiment;

[0054] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0056] In one embodiment, Figure 1 As shown, a method for voice conversion is provided. This embodiment uses the method applied to a computer device as an example. It is understandable that the computer device can be a terminal or a server. In this embodiment, the method includes the following steps:

[0057] S110: Receive the speech to be converted, and extract the speech features and semantic information of the speech to be converted.

[0058] The speech feature is used to represent the sound characteristics of the speech, which can be a speech feature vector. The semantic information is used to represent the linguistic meaning of the speech, that is, the specific meaning expressed by the speech.

[0059] Optionally, the voice to be converted may be voice data stored locally in the computer device, or may be voice data received in real time.

[0060] Optionally, after receiving the speech to be converted, the computer device may use a preset separation model to separate and extract the features and speech of the speech to be converted, thereby obtaining speech features and semantic information of the speech to be converted. The separation model is a pre-trained network model for separating and extracting features and semantic information from speech data.

[0061] S120: Obtain target speech features corresponding to the speech features of the speech to be converted according to a preset speech feature correspondence relationship.

[0062] The speech feature correspondence includes at least one set of speech feature correspondences. Each set of speech feature correspondences includes two speech features, and the corresponding speech feature correspondence is used to indicate that the two speech features correspond to each other and can be converted between them. For example, a speech feature correspondence includes speech feature A and speech feature B. This speech feature correspondence can convert speech feature A into speech feature B, and can also convert speech feature B into speech feature A.

[0063] Optionally, the voice feature correspondence is determined based on the user's voice information input operation before voice conversion and is pre-stored in the computer device. When performing voice conversion, the computer device directly obtains the voice feature correspondence and determines the corresponding target voice feature based on the voice feature correspondence and the voice features of the voice to be converted.

[0064] S130: Generate target converted speech according to target speech features and semantic information of the speech to be converted.

[0065] Optionally, after determining the target speech features and semantic information corresponding to the speech features of the speech to be converted, the computer device combines the target speech features with the semantic information to obtain a target converted speech that retains the semantic information of the speech to be converted but has the target speech features. Specifically, the computer device performs the reverse process of the aforementioned separation and extraction process to obtain a target converted speech that combines the target speech features with the semantic information of the speech to be converted.

[0066] In this embodiment, a computer device obtains the speech to be converted, extracts the speech features and semantic information of the speech to be converted, and then, based on a preset speech feature correspondence, obtains target speech features corresponding to the speech features of the speech to be converted. The computer device then generates the target converted speech based on the target speech features and the semantic information of the speech to be converted. This method achieves conversion from the speech to the target converted speech by replacing only the speech features while retaining the semantic information of the speech to be converted. This method eliminates the need for speech conversion based on a neural network speech conversion model, reduces the computational complexity of the conversion process, and thus shortens the conversion time and improves efficiency.

[0067] In one embodiment, when the speech feature correspondence relationship includes only one set of speech feature correspondence relationships, and the speech feature correspondence relationship represents the correspondence relationship between the first speech feature and the second speech feature, as shown in FIG. Figure 2 As shown, the above S120, obtaining the target speech feature corresponding to the speech feature of the speech to be converted according to the preset speech feature correspondence, includes:

[0068] S210: Calculate the feature similarity between the speech feature of the speech to be converted and the first speech feature and the second speech feature.

[0069] When the speech feature correspondence includes only one set of speech feature correspondences, as described above, the speech feature correspondences can either convert speech feature A into speech feature B or convert speech feature B into speech feature A. In this case, the computer device needs to determine a specific conversion method, that is, whether to convert the speech feature of the speech to be converted into speech feature A or speech feature B. The speech feature A is the first speech feature, and the speech feature B is the second speech feature.

[0070] Optionally, when it is unclear which is the source speech feature and which is the target speech feature in the speech feature correspondence, the computer device may calculate the feature similarity between the speech feature of the speech to be converted and the first speech feature and the second speech feature, respectively, to determine the target speech feature for conversion based on the obtained feature similarity. Taking the speech feature as a speech feature vector as an example, the cosine value between the vectors may be used to characterize the above-mentioned feature similarity. The computer device calculates the cosine value between the speech feature vector of the speech to be converted and the first speech feature vector, and calculates the cosine value between the speech feature vector of the speech to be converted and the second speech feature vector. The larger the cosine value, the greater the feature similarity between the two speech features. Conversely, the smaller the cosine value, the smaller the feature similarity between the two speech features.

[0071] S220: Determine, between the first speech feature and the second speech feature, a speech feature with the minimum feature similarity as a target speech feature.

[0072] Specifically, the computer device determines the target speech feature from the first speech feature and the second speech feature based on the feature similarity obtained above, and specifically uses the speech feature that obtains the minimum feature similarity as the target speech feature. For example, if the cosine value between the speech feature vector of the speech to be converted and the first speech feature vector is greater than the cosine value between the speech feature vector of the speech to be converted and the second speech feature vector, the cosine value between the speech feature vector of the speech to be converted and the second speech feature vector is the minimum feature similarity, and the second speech feature vector that obtains the minimum feature similarity is the target speech feature.

[0073] In this embodiment, when the speech feature correspondences include only one set of speech feature correspondences, to clarify the specific conversion method, the computer device calculates the feature similarity between the speech feature of the speech to be converted and the first and second speech features, and then determines the speech feature with the minimum feature similarity between the first and second speech features as the target speech feature. Determining the target speech feature for conversion based on feature similarity is quick and convenient, and is easy to implement in a computer program, thereby improving speech conversion efficiency.

[0074] In one embodiment, when the speech feature correspondence includes multiple groups of speech feature correspondences, such as Figure 3 As shown, the process of constructing the corresponding relationship of speech features includes:

[0075] S310: Acquire multiple source voices and target voices corresponding to the source voices.

[0076] The speech conversion is to keep the speech information of the source speech and convert the speech features of the source speech into the speech features of the target speech.

[0077] Optionally, the computer device for implementing the above-mentioned voice conversion method is Figure 4 The conversion device shown is used as an example for explanation. The conversion device includes a box shell 1. A mounting top groove 3 is provided on the top of the box shell 1. A microphone rod 4 is mounted inside the mounting top groove 3 via a rotating shaft. A radio microphone 2 is fixedly mounted on one end of the microphone rod 4. A speaker 5 and a connector port 6 are fixedly mounted on the side wall of the box shell 1. A USB interface 7 is provided on the front outer wall of the box shell 1.

[0078] like Figure 4 As shown, the conversion device includes two sets of sound receiving components (M1 and M2). Each set of sound receiving components includes a sound receiving microphone 2 connected to a microphone boom 4. The other end of the microphone boom 4 is connected to the internal circuit of the conversion device. The sound receiving components are used to collect voice data. The internal circuit is used to establish a voice feature correspondence relationship based on the voice data collected by the sound receiving components and realize voice conversion.

[0079] Optionally, the sound receiving component M1 in the conversion device is used to collect source voice, that is, the voice data collected by the sound receiving component M1 is the source voice; the sound receiving component M2 in the conversion device is used to collect target voice, that is, the voice data collected by the sound receiving component M2 is the target voice.

[0080] like Figure 4 As shown, the conversion device includes two sets of input interfaces (E1 and E2), each of which includes at least one USB interface 7. Each USB interface 7 is connected to the internal circuit of the conversion device. The input interfaces are used to receive externally transmitted voice data for use by the internal circuit.

[0081] Optionally, the input interface E1 in the conversion device located on the same side as the sound receiving component M1 is used to input the source voice, that is, the voice data input by the input interface E1 is the source voice; the sound receiving component M2 in the conversion device located on the same side as the sound receiving component M2 is used to input the target voice, that is, the voice data input by the input interface E2 is the target voice.

[0082] Optionally, the speaker 5 on the conversion device is used to play the target converted voice obtained by conversion; the connector 6 is used to output the voice data of the target converted voice to the outside. The conversion device may also include a heat dissipation network to dissipate heat from the entire conversion device.

[0083] When the voice feature correspondence includes a group of voice feature correspondences, the user can use the above-mentioned sound receiving component M1 and sound receiving component M2 to record the source voice and target voice at the same time, or can record them in a certain order so that the internal circuit can extract the features of the source voice and target voice respectively and establish a correspondence between the two.

[0084] When the voice feature correspondence includes multiple groups of voice feature correspondences, the user can use the above-mentioned sound receiving component M1 and sound receiving component M2 to record the source voice and target voice at the same time, and build a group of voice feature correspondences based on the source voice and target voice recorded at the same time, and then record the next group of source voice and target voice, and establish a corresponding set of voice feature correspondences.

[0085] Specifically, a plurality of source voices and target voices corresponding to the source voices can be obtained by utilizing the two groups of sound receiving components on the above-mentioned conversion device.

[0086] S320: Extracting a first speech feature of each source speech and a second speech feature of each target speech.

[0087] Specifically, the sound receiving component in the conversion device transmits each group of source speech and corresponding target speech collected to the internal circuit, and the internal circuit extracts speech features to obtain the first speech feature of the source speech and the second speech feature of the corresponding target speech.

[0088] S330: Associating the corresponding first speech feature with the second speech feature to obtain multiple groups of speech feature correspondences.

[0089] Specifically, the conversion device further associates the corresponding first speech feature and the second speech feature, that is, obtains the correspondence between the two speech features, and as a set of speech feature correspondences, multiple groups of source speech and corresponding target speech can be correspondingly extracted to obtain multiple groups of first speech features and corresponding second speech features. After corresponding association, multiple groups of speech feature correspondences can be obtained.

[0090] In this embodiment, the computer device obtains multiple source voices and target voices corresponding to each source voice, and extracts the first voice features of each source voice and the second voice features of each target voice to associate the corresponding first voice features with the second voice features to obtain multiple groups of voice feature correspondences, thereby realizing the construction of multiple groups of voice feature correspondences and improving the diversity of voice conversion.

[0091] In one embodiment, when the speech feature correspondence includes multiple groups of speech feature correspondences, such as Figure 5 As shown, the above 120, obtaining the target speech feature corresponding to the speech feature of the speech to be converted according to the preset speech feature correspondence, includes:

[0092] S510: Determine a target correspondence relationship in the multiple groups of speech feature correspondence relationships according to a feature similarity between a speech feature of the speech to be converted and a first speech feature in the multiple groups of speech feature correspondence relationships.

[0093] In the case where the speech feature correspondence includes multiple groups of speech feature correspondences, the computer device needs to determine the target correspondences corresponding to the speech features of the speech to be converted, so as to determine the target speech features for conversion according to the target correspondences.

[0094] S520: Determine target speech features according to the target correspondence relationship.

[0095] Specifically, on the basis of clarifying the speech features of the source speech and the speech features of the target speech in each group of speech feature correspondences, the computer device can determine the target correspondence in multiple groups of speech feature correspondences based on the feature similarity between the speech features of the speech to be converted and the speech features of the source speech in multiple groups of speech feature correspondences (i.e., the first speech features), and then determine that the second speech feature in the target correspondence is the above-mentioned target speech feature.

[0096] In an optional embodiment, if Figure 6 As shown, the above-mentioned determination S510, determining the target correspondence relationship in the multiple groups of speech feature correspondence relationships based on the feature similarity between the speech feature of the speech to be converted and the first speech feature in the multiple groups of speech feature correspondence relationships, includes:

[0097] S610: Obtain feature similarity between the speech feature of the speech to be converted and the first speech feature in each group of speech feature correspondences.

[0098] Specifically, each set of speech feature correspondences includes the first speech feature of the source speech and the second speech feature of the target speech. The computer device can calculate the feature similarity between the speech feature of the speech to be converted and the first speech feature in each set of speech feature correspondences, and then determine the target correspondence based on the feature similarity. For example, the speech feature correspondences include three sets of speech feature correspondences, namely: ① the first speech feature S1 corresponds to the second speech feature T1, ② the first speech feature S2 corresponds to the second speech feature T3, and ③ the first speech feature S3 corresponds to the second speech feature T2. The computer device calculates the feature similarity between the speech feature S of the speech to be converted and the first speech features S1, S2, and S3, respectively, and obtains feature similarities C1, C2, and C3 accordingly.

[0099] S620: Determine the first speech feature whose feature similarity is greater than the similarity threshold as the target first speech feature.

[0100] Specifically, the computer compares the obtained feature similarities, determines a feature similarity greater than a similarity threshold, and determines the first speech feature obtained with the feature similarity as the target first speech feature. For example, continuing with the above example, if feature similarity C1 among the obtained feature similarities C1, C2, and C3 is greater than the similarity threshold C, the computer device determines that the first speech feature S1 obtained with the feature similarity C1 is the target first speech feature.

[0101] Optionally, if there are multiple feature similarities among the obtained feature similarities, all of which are greater than the similarity threshold, the first speech feature with the maximum feature similarity is determined as the target first speech feature.

[0102] Optionally, if multiple maximum feature similarities are further included, the computer device extracts the feature representation of the first speech feature with the maximum feature similarity for user selection, and determines the target speech feature based on the user's selection operation.

[0103] S630: Determine a target correspondence relationship in which the first voice feature matches the target first voice feature from among the multiple groups of voice feature correspondence relationships.

[0104] Specifically, after the computer device determines the target first voice feature based on feature similarity, it can determine the voice feature correspondence in which the first voice feature matches the target first voice feature in multiple groups of voice feature correspondences as the target correspondence, such as directly determining the voice feature correspondence in which the first voice feature is identical to the target first voice feature in multiple groups of voice feature correspondences as the target correspondence. Continuing with the above example, if the target first voice feature is the first voice feature S1, the first voice feature S1 in the correspondence ① among the above three groups of voice feature correspondences is the same as the target first voice feature S1, and the computer device can determine that the correspondence ① first voice feature S1 corresponds to the second voice feature T1, which is the above target correspondence.

[0105] In this embodiment, when the voice feature correspondence includes multiple groups of voice feature correspondences, the computer device can determine the target correspondence in the multiple groups of voice feature correspondences based on the feature similarity between the voice feature of the voice to be converted and the first voice feature in the multiple groups of voice feature correspondences, and then determine the target voice feature based on the target correspondence. Specifically, by obtaining the feature similarity between the voice feature of the voice to be converted and the first voice feature in each group of voice feature correspondences, the first voice feature with a feature similarity greater than a similarity threshold is determined as the target first voice feature, so as to determine the target correspondence that matches the first voice feature with the target first voice feature in the multiple groups of voice feature correspondences. In this way, the target correspondence corresponding to the voice feature of the voice to be converted can be quickly and accurately determined in the multiple groups of voice feature correspondences, thereby improving the conversion efficiency and the accuracy of the conversion.

[0106] In one embodiment, in order to improve the accuracy of extracting speech features and semantic information, Figure 7 The above-mentioned step S110 extracts the speech features and semantic information of the speech to be converted, including:

[0107] S710: Input the speech to be converted into a feature extraction model to obtain speech features of the speech to be converted.

[0108] Among them, the feature extraction model is a network model used to extract speech features from speech data, which can be trained using a large amount of speech data.

[0109] Specifically, the computer device may input the speech to be converted into a feature extraction model, and perform speech feature extraction on the speech to be converted through the feature extraction model to obtain speech features of the speech to be converted.

[0110] S720: Input the speech to be converted into a semantic extraction model to obtain semantic information of the speech to be converted.

[0111] Among them, the semantic extraction model is a network model used to extract voice information from voice data, which can be trained using a large amount of voice data.

[0112] Specifically, the computer device may input the speech to be converted into a semantic extraction model, and extract semantic information from the speech to be converted through the semantic extraction model to obtain semantic information of the speech to be converted.

[0113] Optionally, the feature extraction model and semantic extraction model can be two independent models or two sub-models in the same model, and can be extracted in parallel or in a certain order.

[0114] In this embodiment, the computer device extracts the speech features and semantic information of the file to be converted respectively through the feature extraction model and semantic extraction model obtained through training, so as to obtain the speech features and semantic information of the speech to be converted, which can effectively improve the accuracy of the extraction and thus improve the accuracy of the overall speech conversion.

[0115] To facilitate understanding by those skilled in the art, the speech conversion method provided by this application is described in detail below. Figure 8 As shown, the method may include:

[0116] S801: Acquire the speech to be converted, and input the speech to be converted into a feature extraction model to obtain speech features of the speech to be converted; input the speech to be converted into a semantic extraction model to obtain semantic information of the speech to be converted;

[0117] S802: If the speech feature correspondence includes a set of speech feature correspondences, and the speech feature correspondence represents a correspondence between a first speech feature and a second speech feature, then calculating feature similarities between the speech feature of the speech to be converted and the first speech feature and the second speech feature, respectively;

[0118] S803: Determine the speech feature with the minimum feature similarity between the first speech feature and the second speech feature as the target speech feature;

[0119] S804: If the speech feature correspondence includes multiple groups of speech feature correspondences, the process of constructing the speech feature correspondence includes:

[0120] Acquire multiple source voices and target voices corresponding to each source voice; extract first voice features of each source voice and second voice features of each target voice; associate the corresponding first voice features with the second voice features to obtain multiple groups of voice feature correspondences;

[0121] S805: Obtain feature similarity between the speech feature of the speech to be converted and the first speech feature in each group of speech feature correspondences;

[0122] S806: Determine the first speech feature whose feature similarity is greater than a similarity threshold as the target first speech feature;

[0123] S807: Determine a target correspondence relationship in which the first voice feature matches the target first voice feature from among the multiple groups of voice feature correspondence relationships;

[0124] S808, determining target speech features according to the target correspondence;

[0125] S809: Generate target converted speech according to the target speech features and the semantic information of the speech to be converted.

[0126] It should be noted that for the descriptions in S801-S809 above, reference may be made to the relevant descriptions in the above embodiments, and the effects are similar, so this embodiment will not be repeated here.

[0127] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0128] In one embodiment, Figure 9 As shown, a speech conversion device is provided, comprising: a speech processing module 901, a target determination module 902 and a speech generation module 903, wherein:

[0129] The speech processing module 901 is used to obtain the speech to be converted and extract the speech features and semantic information of the speech to be converted;

[0130] The target determination module 902 is configured to obtain target speech features corresponding to speech features of the speech to be converted based on a preset speech feature correspondence relationship; wherein the speech feature correspondence relationship includes at least one group of speech feature correspondence relationships;

[0131] The speech generation module 903 is used to generate target converted speech according to the target speech features and the semantic information of the speech to be converted.

[0132] In one embodiment, if the speech feature correspondence includes a set of speech feature correspondences, and the speech feature correspondences represent a correspondence between a first speech feature and a second speech feature;

[0133] The target determination module 902 is specifically configured to:

[0134] The feature similarities between the speech feature of the speech to be converted and the first speech feature and the second speech feature are calculated respectively; and the speech feature with the minimum feature similarity between the first speech feature and the second speech feature is determined as the target speech feature.

[0135] In one embodiment, if the speech feature correspondence includes multiple groups of speech feature correspondences, the speech processing module 901 is further configured to construct the speech feature correspondences, specifically to:

[0136] Acquire multiple source voices and target voices corresponding to each source voice; extract first voice features of each source voice and second voice features of each target voice; associate the corresponding first voice features with the second voice features to obtain multiple groups of voice feature correspondences.

[0137] In one embodiment, the target determination module 902 is further configured to:

[0138] A target correspondence is determined in the multiple groups of speech feature correspondences according to the feature similarity between the speech feature of the speech to be converted and the first speech feature in the multiple groups of speech feature correspondences; and a target speech feature is determined according to the target correspondence.

[0139] In one embodiment, the target determination module 902 is specifically configured to:

[0140] Obtain feature similarity between the speech feature of the speech to be converted and the first speech feature in each group of speech feature correspondences; determine the first speech feature whose feature similarity is greater than a similarity threshold as the target first speech feature; and determine a target correspondence in multiple groups of speech feature correspondences that matches the first speech feature with the target first speech feature.

[0141] In one embodiment, the speech processing module 901 is specifically configured to:

[0142] The speech to be converted is input into a feature extraction model to obtain the speech features of the speech to be converted; the speech to be converted is input into a semantic extraction model to obtain the semantic information of the speech to be converted.

[0143] Each module in the above-mentioned speech conversion device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0144] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a voice conversion method is implemented.

[0145] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0146] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0147] Acquire the speech to be converted and extract the speech features and semantic information of the speech to be converted; obtain target speech features corresponding to the speech features of the speech to be converted based on a preset speech feature correspondence; wherein the speech feature correspondence includes at least one group of speech feature correspondences; and generate a target converted speech based on the target speech features and the semantic information of the speech to be converted.

[0148] In one embodiment, if the speech feature correspondence includes a set of speech feature correspondences, and the speech feature correspondence represents a correspondence between a first speech feature and a second speech feature; when the processor executes the computer program, the processor further implements the following steps:

[0149] The feature similarities between the speech feature of the speech to be converted and the first speech feature and the second speech feature are calculated respectively; and the speech feature with the minimum feature similarity between the first speech feature and the second speech feature is determined as the target speech feature.

[0150] In one embodiment, if the speech feature correspondence includes multiple groups of speech feature correspondences, the processor further implements the following steps when executing the computer program:

[0151] Acquire multiple source voices and target voices corresponding to each source voice; extract first voice features of each source voice and second voice features of each target voice; associate the corresponding first voice features with the second voice features to obtain multiple groups of voice feature correspondences.

[0152] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0153] A target correspondence is determined in the multiple groups of speech feature correspondences according to the feature similarity between the speech feature of the speech to be converted and the first speech feature in the multiple groups of speech feature correspondences; and a target speech feature is determined according to the target correspondence.

[0154] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0155] Obtain feature similarity between the speech feature of the speech to be converted and the first speech feature in each group of speech feature correspondences; determine the first speech feature whose feature similarity is greater than a similarity threshold as the target first speech feature; and determine a target correspondence in multiple groups of speech feature correspondences that matches the first speech feature with the target first speech feature.

[0156] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0157] The speech to be converted is input into a feature extraction model to obtain the speech features of the speech to be converted; the speech to be converted is input into a semantic extraction model to obtain the semantic information of the speech to be converted.

[0158] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0159] Acquire the speech to be converted and extract the speech features and semantic information of the speech to be converted; obtain target speech features corresponding to the speech features of the speech to be converted based on a preset speech feature correspondence; wherein the speech feature correspondence includes at least one group of speech feature correspondences; and generate a target converted speech based on the target speech features and the semantic information of the speech to be converted.

[0160] In one embodiment, if the speech feature correspondence includes a set of speech feature correspondences, and the speech feature correspondences represent a correspondence between a first speech feature and a second speech feature; when the computer program is executed by a processor, the following steps are further implemented:

[0161] The feature similarities between the speech feature of the speech to be converted and the first speech feature and the second speech feature are calculated respectively; and the speech feature with the minimum feature similarity between the first speech feature and the second speech feature is determined as the target speech feature.

[0162] In one embodiment, if the speech feature correspondence includes multiple groups of speech feature correspondences, the computer program further implements the following steps when executed by the processor:

[0163] Acquire multiple source voices and target voices corresponding to each source voice; extract first voice features of each source voice and second voice features of each target voice; associate the corresponding first voice features with the second voice features to obtain multiple groups of voice feature correspondences.

[0164] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0165] A target correspondence is determined in the multiple groups of speech feature correspondences according to the feature similarity between the speech feature of the speech to be converted and the first speech feature in the multiple groups of speech feature correspondences; and a target speech feature is determined according to the target correspondence.

[0166] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0167] Obtain feature similarity between the speech feature of the speech to be converted and the first speech feature in each group of speech feature correspondences; determine the first speech feature whose feature similarity is greater than a similarity threshold as the target first speech feature; and determine a target correspondence in multiple groups of speech feature correspondences that matches the first speech feature with the target first speech feature.

[0168] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0169] The speech to be converted is input into a feature extraction model to obtain the speech features of the speech to be converted; the speech to be converted is input into a semantic extraction model to obtain the semantic information of the speech to be converted.

[0170] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0171] Acquire the speech to be converted and extract the speech features and semantic information of the speech to be converted; obtain target speech features corresponding to the speech features of the speech to be converted based on a preset speech feature correspondence; wherein the speech feature correspondence includes at least one group of speech feature correspondences; and generate a target converted speech based on the target speech features and the semantic information of the speech to be converted.

[0172] In one embodiment, if the speech feature correspondence includes a set of speech feature correspondences, and the speech feature correspondences represent a correspondence between a first speech feature and a second speech feature; when the computer program is executed by a processor, the following steps are further implemented:

[0173] The feature similarities between the speech feature of the speech to be converted and the first speech feature and the second speech feature are calculated respectively; and the speech feature with the minimum feature similarity between the first speech feature and the second speech feature is determined as the target speech feature.

[0174] In one embodiment, if the speech feature correspondence includes multiple groups of speech feature correspondences, the computer program further implements the following steps when executed by the processor:

[0175] Acquire multiple source voices and target voices corresponding to each source voice; extract first voice features of each source voice and second voice features of each target voice; associate the corresponding first voice features with the second voice features to obtain multiple groups of voice feature correspondences.

[0176] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0177] A target correspondence is determined in the multiple groups of speech feature correspondences according to the feature similarity between the speech feature of the speech to be converted and the first speech feature in the multiple groups of speech feature correspondences; and a target speech feature is determined according to the target correspondence.

[0178] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0179] Obtain feature similarity between the speech feature of the speech to be converted and the first speech feature in each group of speech feature correspondences; determine the first speech feature whose feature similarity is greater than a similarity threshold as the target first speech feature; and determine a target correspondence in multiple groups of speech feature correspondences that matches the first speech feature with the target first speech feature.

[0180] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0181] The speech to be converted is input into a feature extraction model to obtain the speech features of the speech to be converted; the speech to be converted is input into a semantic extraction model to obtain the semantic information of the speech to be converted.

[0182] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0183] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0184] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A voice conversion method, characterized in that: The method comprises: Acquiring the speech to be converted, and extracting speech features and semantic information of the speech to be converted; According to a preset speech feature correspondence relationship, obtaining a target speech feature corresponding to the speech feature of the speech to be converted; wherein the speech feature correspondence relationship includes at least one group of speech feature correspondence relationships; Combining the target speech feature with the semantic information of the speech to be converted to obtain a target converted speech that retains the semantic information of the speech to be converted and has the target speech feature; If the speech feature correspondence includes a set of speech feature correspondences, and the speech feature correspondences represent a correspondence between a first speech feature and a second speech feature, then obtaining, based on the preset speech feature correspondences, a target speech feature corresponding to the speech feature of the speech to be converted includes: Calculating feature similarities between the speech feature of the speech to be converted and the first speech feature and the second speech feature respectively; Determining the speech feature with the minimum feature similarity between the first speech feature and the second speech feature as the target speech feature; If the speech feature correspondence includes multiple groups of speech feature correspondences, then obtaining the target speech feature corresponding to the speech feature of the speech to be converted according to the preset speech feature correspondence includes: Obtaining feature similarity between the speech feature of the speech to be converted and the first speech feature in each group of speech feature correspondences; Determining the first speech feature whose feature similarity is greater than a similarity threshold as a target first speech feature; Determining a target correspondence relationship in which a first voice feature matches the target first voice feature among the multiple groups of voice feature correspondence relationships; The target speech feature is determined according to the target correspondence.

2. The method according to claim 1, characterized in that If the speech feature correspondence includes multiple groups of speech feature correspondences, the process of constructing the speech feature correspondence includes: Acquire multiple source voices and target voices corresponding to the source voices; Extracting a first speech feature of each source speech and a second speech feature of each target speech; The corresponding first speech feature and second speech feature are associated to obtain the multiple groups of speech feature correspondences.

3. The method according to claim 1 or 2, characterized in that The extracting of speech features and semantic information of the speech to be converted includes: Inputting the speech to be converted into a feature extraction model to obtain speech features of the speech to be converted; The speech to be converted is input into a semantic extraction model to obtain semantic information of the speech to be converted.

4. The method according to claim 1 or 2, characterized in that The determining that the first speech feature having the feature similarity greater than a similarity threshold is the target first speech feature includes: In the case that there are multiple feature similarities all greater than the similarity threshold, the first speech feature corresponding to the largest feature similarity is used as the target first speech feature.

5. The method according to claim 4, characterized in that The method further comprises: In the case that there are multiple maximum feature similarities, the target speech feature is determined from the first speech features corresponding to the maximum feature similarities based on a user selection operation.

6. A voice conversion device, characterized in that: The device comprises: A speech processing module, configured to obtain speech to be converted and extract speech features and semantic information of the speech to be converted; A target determination module, configured to obtain target speech features corresponding to the speech features of the speech to be converted according to a preset speech feature correspondence relationship; wherein the speech feature correspondence relationship includes at least one group of speech feature correspondence relationships; a speech generation module, configured to combine the target speech features with the semantic information of the speech to be converted, to obtain a target converted speech having the target speech features while retaining the semantic information of the speech to be converted; If the speech feature correspondence includes a group of speech feature correspondences, and the speech feature correspondence represents a correspondence between a first speech feature and a second speech feature, the target determination module is used to calculate the feature similarity between the speech feature of the speech to be converted and the first speech feature and the second speech feature respectively; the speech feature with the minimum feature similarity between the first speech feature and the second speech feature is determined as the target speech feature. If the speech feature correspondence includes multiple groups of speech feature correspondences, the target determination module is used to obtain the feature similarity between the speech feature of the speech to be converted and the first speech feature in each group of speech feature correspondences; determine the first speech feature whose feature similarity is greater than the similarity threshold as the target first speech feature; determine the target correspondence between the first speech feature and the target first speech feature in the multiple groups of speech feature correspondences; and determine the target speech feature based on the target correspondence.

7. The device according to claim 6, characterized in that The target determination module is also used to obtain multiple source voices and target voices corresponding to each of the source voices; extract the first voice features of each of the source voices and the second voice features of each of the target voices; and associate the corresponding first voice features and second voice features to obtain the multiple groups of voice feature correspondences.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images