Voice changing method, voice changing system, electronic device and storage medium

Through the streaming voice change method, the calculation of Meer spectrum and semantic features is used to solve the problem of voice change delay in the prior art, real-time voice change of voice signals is realized, and the voice change effect is improved.

CN114187918BActive Publication Date: 2024-11-26HARBIN INST OF TECH SHENZHEN GRADUATE SCHOOL
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111441755.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-30
Publication Date
2024-11-26
Estimated Expiration
2041-11-30

AI Technical Summary

Technical Problem

The existing voice-changing technology requires obtaining a complete voice signal before voice-changing conversion, resulting in delayed use and affecting the voice-changing effect.

Method used

The streaming voice change method is adopted to calculate the Mel spectrum and semantic features of the speech signal, and the preset frame number and semantic encoder are used to generate semantic features, avoid relying on complete voice fragments, realize streaming calculation and voice change, and reduce voice change delay.

Benefits of technology

It realizes streaming voice change based on semantic features and speaker features, reducing voice change delay and improving the real-time sound change effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114187918B_ABST
    Figure CN114187918B_ABST
Patent Text Reader

Abstract

The present invention discloses a voice changing method, a voice changing system, an electronic device and a storage medium. The voice changing method includes: obtaining an original speaker's voice signal; calculating a first Mel spectrum according to the original speaker's voice signal; determining that the number of newly added frames in the first Mel spectrum is equal to a preset number of frames, and identifying the newly added number of frames as a target feature block; obtaining an associated feature block at the n-1th time step according to the nth time step of the target feature block; wherein the number of associated frames in the associated feature block is equal to the preset number of frames, and n is a positive integer greater than or equal to 1; obtaining semantic features according to the target feature block, the associated feature block and a preset semantic encoder; obtaining a target speaker's voice signal; obtaining speaker features according to the target speaker's voice signal; performing a splicing operation on the semantic features and the speaker features to obtain a second Mel spectrum; and obtaining a voice changing signal according to the second Mel spectrum. The present application can realize streaming voice changing, thereby reducing the voice changing delay to a certain extent.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of voice changing technology, and in particular to a voice changing method, a voice changing system, an electronic device and a storage medium. Background Art

[0002] A voice changer refers to a device that can convert the voice signal of an original speaker into the voice signal of a target speaker.

[0003] In the related art, the voice changer needs to obtain the complete speech signal of the original speaker to perform the voice change conversion, which causes a certain usage delay in the voice change conversion, thus affecting the voice change effect.

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a voice changing method, a voice changing system, an electronic device and a storage medium, which can realize streaming voice changing, thereby reducing the voice changing delay to a certain extent.

[0005] According to the first aspect of the present invention, the voice changing method includes: obtaining an original speaker's speech signal; calculating a first Mel spectrum according to the original speaker's speech signal; determining that the number of newly added frames in the first Mel spectrum is equal to a preset number of frames, and identifying the newly added number of frames as a target feature block; obtaining an associated feature block at the n-1th time step according to the nth time step of the target feature block; wherein the number of associated frames in the associated feature block is equal to the preset number of frames, and n is a positive integer greater than or equal to 1; obtaining semantic features according to the target feature block, the associated feature block and a preset semantic encoder; obtaining a target speaker's speech signal; obtaining speaker features according to the target speaker's speech signal; performing a splicing operation on the semantic features and the speaker features, and obtaining a second Mel spectrum according to the semantic features after the splicing operation, the speaker features and a preset decoder; and obtaining a voice changing signal according to the second Mel spectrum and a preset vocoder.

[0006] The voice changing method according to the embodiment of the present invention has at least the following beneficial effects: the newly added frames in the first Mel spectrum are identified as target feature blocks according to the preset frame number. The semantic features are obtained according to the target feature block corresponding to the current time step (i.e., the nth time step), the associated feature block corresponding to the "previous" time step (i.e., the n-1th time step), and the preset semantic encoder, that is, the voice changing method provided by the embodiment of the present application does not rely on the "below" (i.e., the feature block corresponding to the n+1th time step) to calculate the semantic features. Therefore, the embodiment of the present application can realize the streaming calculation of semantic features, avoid the need for complete speech segments to generate semantic features, and then realize streaming voice changing according to semantic features and speaker features, and reduce the voice changing delay to a certain extent.

[0007] According to some embodiments of the present invention, the semantic encoder includes a first self-attention unit; the obtaining of semantic features based on the target feature block, the associated feature block and a preset semantic encoder includes: obtaining a semantic modeling unit probability distribution based on the target feature block, the associated feature block and the first self-attention unit; and obtaining the semantic features based on the semantic modeling unit probability distribution.

[0008] According to some embodiments of the present invention, before the concatenation operation is performed on the semantic features and the speaker features, the voice changing method further includes: performing a first mutual information minimization operation on the semantic features and the speaker features; obtaining a baseband signal based on the original speaker speech signal; and performing a second mutual information minimization operation on the speaker features and the baseband signal.

[0009] According to some embodiments of the present invention, the voice changing method further includes: obtaining a prosodic feature based on the first Mel spectrum and a preset prosodic encoder; performing a third mutual information minimization operation on the prosodic feature and the speaker feature; the splicing operation on the semantic feature and the speaker feature, and generating a second Mel spectrum based on the semantic feature after the splicing operation, the speaker feature and a preset decoder, including: performing a splicing operation on the semantic feature, the speaker feature and the prosodic feature, and generating the second Mel spectrum based on the semantic feature after the splicing operation, the speaker feature, the prosodic feature and the decoder.

[0010] According to some embodiments of the present invention, the decoder includes a second self-attention unit; the concatenation operation is performed on the semantic features and the speaker features, and a second Mel spectrum is generated based on the semantic features and the speaker features after the concatenation operation and a preset decoder, including: the second self-attention unit performs reconstruction loss training on the semantic features and the speaker features to generate the second Mel spectrum.

[0011] According to a second aspect of the present invention, the voice changing system comprises: an audio acquisition module for acquiring an original speaker voice signal; a voice changing calculation module, the voice changing module comprising a conversion module, a semantic encoder and a target speaker encoder, the conversion module being used to calculate a first Mel spectrum according to the original speaker voice signal, the semantic encoder being used to determine that the number of newly added frames in the first Mel spectrum is equal to a preset number of frames, and identifying the newly added number of frames as a target feature block; obtaining an associated feature block at an n-1th time step according to the nth time step of the target feature block; wherein the number of associated frames in the associated feature block is equal to the preset number of frames, and n is a positive integer greater than or equal to 1; obtaining semantic features according to the target feature block, the associated feature block and a preset semantic encoder; the target speaker encoder being used to obtain speaker features according to the target speaker voice signal; a decoder being used to generate a second Mel spectrum according to the semantic features and the speaker features after a splicing operation; and a vocoder being used to obtain a voice changing signal according to the second Mel spectrum.

[0012] According to some embodiments of the present invention, the voice change calculation module includes: a prosody encoder, configured to obtain prosody features according to the first Mel-spectrogram.

[0013] According to some embodiments of the present invention, the voice changing calculation module further includes: a decoupling unit, configured to perform a first mutual information minimization operation on the semantic feature and the speaker feature, perform a second mutual information minimization operation on the speaker feature and the baseband signal, and perform a third mutual information minimization operation on the prosodic feature and the speaker feature.

[0014] The electronic device according to the third aspect of the embodiment of the present invention is characterized in that it includes: at least one processor; at least one memory for storing at least one program; when the at least one program is executed by the at least one processor, the at least one processor implements the voice changing method as described in the first aspect.

[0015] According to the computer-readable storage medium of the fourth aspect of the present invention, processor-executable instructions are stored therein, and it is characterized in that the processor-executable instructions are used to implement the voice changing method as described in the first aspect when executed by the processor.

[0016] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments, wherein:

[0018] Figure 1A schematic diagram of a process of a voice changing method according to an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a target feature block and an associated feature block according to an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a process of a voice changing method according to an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of a process of a voice changing method according to an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of a process of a voice changing method according to an embodiment of the present invention;

[0023] Figure 6 FIG. 4 is a schematic diagram of a voice changing system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0025] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0026] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0027] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0028] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0029] Reference Figure 1 , the present application embodiment provides a voice changing method. The voice changing method includes but is not limited to steps S110 to S190:

[0030] S110, obtaining a voice signal of an original speaker;

[0031] S120, calculating and obtaining a first Mel spectrum according to the original speaker's speech signal;

[0032] Specifically, the original speaker represents the user, and the target speaker represents the audio voice-changing object that the user expects to obtain through the voice-changing method. The audio signal of the user is collected by the audio acquisition module to obtain the original speaker's voice signal. The original speaker's voice signal is subjected to acoustic feature changes to obtain a first Mel spectrum. For example, the original speaker's voice signal is converted into a frequency domain signal by discrete Fourier transform, the frequency domain signal is converted into a Mel scale according to the Mel scale, and the Mel scale is logarithmized to obtain the first Mel spectrum.

[0033] S130, determining that the number of newly added frames in the first Mel-spectrum spectrum is equal to a preset number of frames, and identifying the newly added frames as a target feature block;

[0034] S140, obtaining an associated feature block at the n-1th time step according to the nth time step of the target feature block; wherein the number of associated frames in the associated feature block is equal to a preset number of frames, and n is a positive integer greater than or equal to 1;

[0035] Specifically, in order to achieve streaming conversion of the original speaker's speech signal, a preset number of frames for single inference is set. When the number of newly added frames in the first Mel spectrum is equal to the preset number of frames, the Mel spectrum features corresponding to the newly added frames are identified as a target feature block, that is, the first Mel spectrum is divided into units of the preset number of frames. For example, referring to Figure 2, assuming that the preset number of frames is two frames, each of the two newly added frames of signals in the first Mel spectrum is identified as a target feature block. Among them, one target feature block corresponds to one time step. Therefore, at the first time step, the target feature block includes the Mel spectrum features corresponding to the first frame signal and the second frame signal, and the associated feature block is "empty" at this time; at the second time step, the target feature block includes the Mel spectrum features corresponding to the third frame signal and the fourth frame signal, and the associated feature block includes the Mel spectrum features corresponding to the first frame signal and the second frame signal; at the third time step, the target feature block includes the Mel spectrum features corresponding to the fifth frame signal and the sixth frame signal, and the associated feature block includes the Mel spectrum features corresponding to the third frame signal and the fourth frame signal; at the fourth time step, the target feature block includes the Mel spectrum features corresponding to the seventh frame signal and the eighth frame signal, and the associated feature block includes the Mel spectrum features corresponding to the fifth frame signal and the sixth frame signal. It can be understood that the specific value of the preset number of frames can be adaptively adjusted according to the accuracy of the semantic features, and the embodiments of the present application are not specifically limited.

[0036] S150, obtaining semantic features according to the target feature block, the associated feature block and a preset semantic encoder;

[0037] Specifically, the target feature block and the associated feature block are used as input signals of the preset semantic encoder to obtain semantic features. That is, the acquisition of semantic features does not depend on the "context", but is only obtained based on the target feature block corresponding to the current time step and the adjacent "previous context" (i.e., the associated feature block) to achieve streaming semantic feature encoding of the original speaker's speech signal. It can be understood that when the target feature block corresponding to the first time step (i.e., the Mel-spectrogram features corresponding to the first frame signal and the second frame signal) is used as the input signal of the semantic encoder, that is, when the value of n is 1, the "previous context" content is empty. At this time, the semantic features corresponding to the first time step only need to be obtained based on the Mel-spectrogram features corresponding to the first frame signal and the second frame signal.

[0038] S160, obtaining a target speaker's voice signal;

[0039] S170, obtaining speaker features according to the target speaker's speech signal;

[0040] Specifically, a voice change object database is set up, which stores multiple candidate speaker voice signals, and a target speaker voice signal is obtained from the multiple candidate speaker voice signals according to the user's voice change selection. The target speaker voice signal is encoded to obtain speaker features.

[0041] S180, performing a concatenation operation on the semantic features and the speaker features, and generating a second Mel-spectrogram according to the concatenated semantic features, the speaker features and a preset decoder;

[0042] S190: Obtain a voice-changing signal according to the second Mel-spectrogram and a preset vocoder.

[0043] Specifically, the semantic features and speaker features obtained according to the above steps are concatenated, and the decoder obtains a second Mel spectrum according to the concatenated semantic features and speaker features. Then, the vocoder converts the second Mel spectrum into a corresponding voice-changing signal, thereby changing the original speaker's voice signal into a voice-changing signal with the target speaker as the object.

[0044] The voice changing method provided in the embodiment of the present application identifies the newly added frames in the first Mel spectrum as the target feature block according to the preset frame number. The semantic feature is obtained according to the target feature block corresponding to the current time step (i.e., the nth time step), the associated feature block corresponding to the "previous" time step (i.e., the n-1th time step), and the preset semantic encoder, that is, the voice changing method provided in the embodiment of the present application does not rely on the "below" (i.e., the feature block corresponding to the n+1th time step) to calculate the semantic feature. Therefore, the embodiment of the present application can realize the streaming calculation of the semantic feature, avoiding the need for a complete speech segment to generate the semantic feature, thereby realizing the streaming voice changing according to the semantic feature and the speaker feature, and reducing the voice changing delay to a certain extent.

[0045] Reference Figure 3 In some embodiments, the semantic encoder includes a first self-attention unit. Step S150 includes but is not limited to sub-steps S310 to S320:

[0046] S310, obtaining a semantic modeling unit probability distribution according to the target feature block, the associated feature block and the first self-attention unit;

[0047] S320, obtaining semantic features according to the probability distribution of semantic modeling units.

[0048] Specifically, the target feature block and the associated feature block are used as input signals of the first self-attention unit, and after feature transformation according to the attention mechanism, the probability distribution of the semantic modeling unit is obtained, and the probability distribution of the semantic modeling unit is used as the semantic feature. For example, taking the second time step as an example, the target feature block includes the Mel spectrum features corresponding to the third frame signal and the fourth frame signal, and the associated feature block includes the Mel spectrum features corresponding to the first frame signal and the second frame signal. The first self-attention unit performs phoneme classification on the first frame signal to the fourth frame signal according to softmax, and obtains the phoneme probability of each frame signal, and obtains the probability distribution of the semantic modeling unit at the second time step according to the phoneme probability corresponding to the third frame signal and the fourth frame signal, that is, the semantic feature of the second time step is obtained. It can be understood that the calculation method of the probability distribution of the semantic modeling unit can also be adaptively adjusted according to actual needs, and the embodiments of the present application are not specifically limited.

[0049] Reference Figure 4 In some embodiments, before step S180, the voice changing method further includes the steps of:

[0050] S410, performing a first mutual information minimization operation on the semantic features and the speaker features;

[0051] S420, obtaining a baseband signal according to the original speaker's voice signal;

[0052] S430: Perform a second mutual information minimization operation on the speaker feature and the baseband signal.

[0053] Specifically, a fundamental frequency extractor is used to extract fundamental frequency features from original speaker features, and semantic features are fully decoupled from speaker features (i.e., a first mutual information minimization operation is performed), and speaker features are fully decoupled from fundamental frequency features (i.e., a second mutual information minimization operation is performed). This ensures that when the voice-changing object is changed, i.e., a new target speaker speech signal is obtained from multiple candidate speaker speech signals, there is no need to adjust and train the corresponding decoder, thereby improving the adaptability of the voice-changing method.

[0054] It is understandable that the semantic encoder can be trained using a labeled data set and using the CTC loss function, so that the semantic encoder can extract the semantic features in the corresponding target feature block and filter out the paralinguistic features (such as prosodic features, etc.) in the target feature block. The first self-attention unit generates a semantic modeling unit probability distribution based on the semantic features.

[0055] It is understandable that when the semantic encoder is trained, the parameters of the semantic encoder may be frozen to extract accurate semantic features, so that the first mutual information minimization operation can be fully performed later and the training time can be reduced to a certain extent.

[0056] Reference Figure 5 In some embodiments, the voice changing method further comprises the steps of:

[0057] S510, obtaining a prosody feature according to the first Mel spectrum and a preset prosody encoder;

[0058] S520: Perform a third mutual information minimization operation on the prosodic features and the speaker features.

[0059] Step S180 includes sub-steps:

[0060] A concatenation operation is performed on the semantic features, speaker features, and prosodic features, and a second mel spectrum is generated according to the concatenated semantic features, speaker features, prosodic features, and a decoder.

[0061] Specifically, in order to improve the naturalness of the voice-changed signal, the prosodic information such as the base frequency, the length of the speech unit, the pause, and the energy in the first Mel spectrum can also be encoded by a prosodic encoder to obtain prosodic features. The prosodic features are fully decoupled from the speaker features, that is, the third mutual information minimization operation is performed, and the decoder obtains the second Mel spectrum based on the fully decoupled and spliced ​​semantic features, speaker features, and prosodic features, so that the voice changer can obtain a voice-changed signal with high naturalness and streaming voice change according to the second Mel spectrum, that is, the voice change method provided in the embodiment of the present application can achieve the function of retaining the semantic features and prosodic features in the original speaker's voice signal, but changing the speaker's timbre and style to the voice-changed object (that is, the target speaker).

[0062] It is understandable that the prosody encoder can use the VAE algorithm to encode prosody information, which includes displayable information and non-displayable information. Displayable information such as fundamental frequency and energy can be directly calculated and encoded; for non-displayable information such as the length of speech units and pauses, the dimensions of the key layer of the neural network need to be controlled so that the prosody encoder only retains the key information, thereby realizing the extraction and encoding of the non-displayable information.

[0063] In some embodiments, step S180 includes sub-steps:

[0064] The second self-attention unit performs reconstruction loss training on the semantic features and speaker features to generate the second Mel spectrum;

[0065] Specifically, the decoder includes a second self-attention unit, and the second self-attention unit performs reconstruction loss training on the splicing features of semantic features and speaker features according to the self-attention mechanism, that is, performs feature transformation to obtain a second Mel spectrum, so that the vocoder can generate a corresponding voice-changing signal according to the second Mel spectrum. It can be understood that when the features extracted from the first Mel spectrum include prosodic features, the second self-attention unit generates a corresponding second Mel spectrum according to the semantic features, speaker features and prosodic features, that is, the second self-attention unit performs reconstruction loss training on the semantic features, speaker features and prosodic features to obtain the corresponding second Mel spectrum, and the vocoder can generate a highly natural voice-changing signal according to the second Mel spectrum.

[0066] Reference Figure 6 The embodiment of the present application also provides a voice changing system. The voice changing system includes:

[0067] The audio collection module 100 is used to obtain the original speaker's voice signal;

[0068] The voice change calculation module 200 includes a semantic encoder 210, a target speaker encoder 220 and a conversion module 230. The conversion module 230 is used to calculate a first Mel spectrum according to the original speaker's speech signal. The semantic encoder 210 is used to determine that the number of newly added frames in the first Mel spectrum is equal to a preset number of frames, and the newly added frames are identified as a target feature block; according to the nth time step of the target feature block, an associated feature block of the n-1th time step is obtained; wherein the number of associated frames in the associated feature block is equal to the preset number of frames, and n is a positive integer greater than or equal to 1; semantic features are obtained according to the target feature block, the associated feature block and the preset semantic encoder 210; the target speaker encoder 220 is used to obtain speaker features according to the target speaker's speech signal;

[0069] Decoder 300, used to generate a second Mel spectrum according to the semantic features and speaker features after the concatenation operation;

[0070] The vocoder 400 is used to obtain a voice-changed signal according to the second Mel-spectrogram.

[0071] The voice change calculation module 200 further includes a prosody encoder 240 and a decoupling unit 250. The prosody encoder 240 is used to obtain prosody features according to the first Mel spectrum; the decoupling unit 250 is used to perform a first mutual information minimization operation on the semantic features and the speaker features, a second mutual information minimization operation on the speaker features and the baseband signal, and a third mutual information minimization operation on the prosody features and the speaker features.

[0072] It can be seen that the contents of the above-mentioned voice changing method embodiment are all applicable to the embodiment of the present voice changing system. The functions specifically implemented by the present voice changing system embodiment are the same as those of the above-mentioned voice changing method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned voice changing method embodiment.

[0073] The embodiment of the present application also provides an electronic device, which includes: at least one processor, and a memory connected to the at least one processor in communication. The memory stores instructions, and the instructions are executed by the at least one processor, so that when the at least one processor executes the instructions, the voice changing method described in any of the above embodiments is implemented.

[0074] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the voice changing method described in any of the above embodiments.

[0075] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0076] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0077] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in the relevant technical field without departing from the purpose of the present application. In addition, the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

Claims

1. A voice changing method, characterized in that: include: Obtaining the original speaker's voice signal; Calculate a first Mel spectrum according to the original speaker's speech signal; Determine that the number of newly added frames in the first Mel-spectrum spectrum is equal to a preset number of frames, and identify the newly added frames as a target feature block; According to the nth time step of the target feature block, an associated feature block of the n-1th time step is obtained; wherein the number of associated frames in the associated feature block is equal to the preset number of frames, and n is a positive integer greater than or equal to 1; Obtaining semantic features according to the target feature block, the associated feature block and a preset semantic encoder; Acquire the target speaker's speech signal; Obtain speaker features according to the target speaker's speech signal; Performing a concatenation operation on the semantic features and the speaker features, and generating a second Mel-spectrogram according to the concatenated semantic features, the speaker features and a preset decoder; A voice-changed signal is obtained according to the second Mel-spectrogram and a preset vocoder.

2. The voice changing method according to claim 1, characterized in that: The semantic encoder comprises a first self-attention unit; The obtaining of semantic features according to the target feature block, the associated feature block and a preset semantic encoder includes: Obtaining a semantic modeling unit probability distribution according to the target feature block, the associated feature block and the first self-attention unit; The semantic feature is obtained according to the probability distribution of the semantic modeling unit.

3. The voice changing method according to claim 2, characterized in that: Before the splicing operation is performed on the semantic features and the speaker features, the voice changing method further includes: Performing a first mutual information minimization operation on the semantic feature and the speaker feature; Obtaining a baseband signal according to the original speaker's speech signal; A second mutual information minimization operation is performed on the speaker feature and the baseband signal.

4. The voice changing method according to claim 3, characterized in that: The voice changing method further comprises: Obtaining a prosody feature according to the first Mel-spectrogram and a preset prosody encoder; performing a third mutual information minimization operation on the prosodic feature and the speaker feature; The step of performing a splicing operation on the semantic features and the speaker features, and generating a second Mel spectrum according to the semantic features and the speaker features after the splicing operation and a preset decoder, includes: A concatenation operation is performed on the semantic feature, the speaker feature, and the prosodic feature, and the second Mel-spectrogram is generated according to the concatenated semantic feature, the speaker feature, the prosodic feature, and the decoder.

5. The voice changing method according to any one of claims 1 to 3, characterized in that: The decoder comprises a second self-attention unit; The step of performing a splicing operation on the semantic features and the speaker features, and generating a second Mel spectrum according to the semantic features and the speaker features after the splicing operation and a preset decoder, includes: The second self-attention unit performs reconstruction loss training on the semantic features and the speaker features to generate the second Mel spectrum.

6. Voice changing system, characterized in that: include: An audio acquisition module is used to obtain the original speaker's voice signal; A voice change calculation module, the voice change module includes a conversion module, a semantic encoder and a target speaker encoder, the conversion module is used to calculate a first Mel spectrum according to the original speaker voice signal, the semantic encoder is used to determine that the number of newly added frames in the first Mel spectrum is equal to a preset number of frames, and the newly added number of frames is identified as a target feature block; according to the nth time step of the target feature block, an associated feature block of the n-1th time step is obtained; wherein the number of associated frames in the associated feature block is equal to the preset number of frames, and n is a positive integer greater than or equal to 1; semantic features are obtained according to the target feature block, the associated feature block and a preset semantic encoder; the target speaker encoder is used to obtain speaker features according to the target speaker voice signal; A decoder, used for generating a second Mel spectrum according to the semantic features and the speaker features after the concatenation operation; A vocoder is used to obtain a voice-changed signal according to the second mel-spectrogram.

7. The voice changing system according to claim 6, characterized in that: The voice changing calculation module comprises: A prosody encoder is used to obtain a prosody feature according to the first Mel-spectrogram.

8. The voice changing system according to claim 7, characterized in that: The voice changing calculation module also includes: The decoupling unit is used to perform a first mutual information minimization operation on the semantic feature and the speaker feature, perform a second mutual information minimization operation on the speaker feature and the baseband signal, and perform a third mutual information minimization operation on the prosodic feature and the speaker feature.

9. An electronic device, characterized in that include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the voice changing method according to any one of claims 1 to 5.

10. A computer-readable storage medium having processor-executable instructions stored therein, characterized in that: The processor-executable instructions are used to implement the voice changing method according to any one of claims 1 to 5 when executed by the processor.