Voice conversion method and device, electronic equipment and storage medium

Abstract

The embodiment of the application provides a voice conversion method and device, electronic equipment and storage medium, and belongs to the technical field of digital medical treatment. The method comprises the following steps: obtaining sample voice data; obtaining a sample timbre vector through an original timbre encoder; obtaining a sample content vector through an original content encoder; obtaining a sample object feature vector through an original object feature encoder; obtaining sample voice reconstruction data through an original decoder; performing voice comparison according to the sample voice data and the sample voice reconstruction data to obtain voice loss data; obtaining decoupling loss data from the sample timbre vector, the sample content vector and the sample object feature vector; obtaining a target voice model from the original voice model according to the voice loss data and the decoupling loss data; and performing voice conversion on obtained target voice data through the target voice model and preset target conversion object data to obtain converted voice data. The embodiment of the application can improve the accuracy of voice conversion.

Description

Technical Field

[0001] This application relates to the field of digital medical technology, and in particular to a speech conversion method and apparatus, electronic device and storage medium. Background Technology

[0002] Speech conversion refers to converting the speech of a source speaker into the speech of a target speaker without altering the content of the source speaker's speech. For example, in digital healthcare scenarios, during intelligent diagnosis, remote consultations, and online consultations, speech conversion can be performed on the speech of the person being consulted to protect their privacy, such as converting the speech of the person being consulted (Y) into the speech of the person being consulted (Y). In related technologies, the speech of the source speaker (i.e., the person being consulted) includes speech content and different speech representation vectors. The correlation between these different speech representation vectors and speech content leads to low accuracy when directly converting speech based on the speech characteristics of the target speaker (i.e., Y). Therefore, improving the accuracy of speech conversion has become an urgent technical problem to be solved. Summary of the Invention

[0003] The main objective of this application is to provide a speech conversion method, apparatus, electronic device, and storage medium, aiming to improve the accuracy of the speech conversion method.

[0004] To achieve the above objectives, a first aspect of this application proposes a speech conversion method, the method comprising:

[0005] Acquire sample speech data;

[0006] The sample speech data is input into a preset original speech model; wherein, the original speech model includes an original timbre encoder, an original content encoder, an original object feature encoder, and an original decoder;

[0007] The original timbre encoder is used to perform timbre encoding on the sample speech data to obtain a sample timbre vector.

[0008] The original content encoder performs content encoding on the sample speech data to obtain a sample content vector;

[0009] The original object feature encoder is used to perform object feature encoding on the sample speech data to obtain the sample object feature vector;

[0010] The original decoder performs speech reconstruction processing on the sample timbre vector, the sample content vector, and the sample object feature vector to obtain sample speech reconstruction data.

[0011] Speech comparison is performed based on the sample speech data and the sample speech reconstruction data to obtain speech loss data;

[0012] The sample timbre vector, the sample content vector, and the sample object feature vector are decoupled to obtain decoupling loss data.

[0013] The original speech model is adjusted according to the speech loss data and the decoupling loss data to obtain the target speech model.

[0014] The acquired target speech data is converted into speech data by using the target speech model and preset target conversion object data.

[0015] In some embodiments, the step of performing content encoding processing on the sample speech data using the original content encoder to obtain a sample content vector includes:

[0016] The sample speech data is subjected to speech perturbation processing to obtain perturbed speech data;

[0017] The perturbation speech data is processed by the original content encoder to obtain the sample content vector.

[0018] In some embodiments, the step of performing speech perturbation processing on the sample speech data to obtain perturbed speech data includes:

[0019] The pitch of the sample speech data is extracted to obtain the original pitch data;

[0020] The original pitch data is sorted to obtain a pitch sequence;

[0021] Median feature extraction is performed on the pitch sequence to obtain candidate pitch data;

[0022] The candidate pitch data is adjusted to obtain the target pitch data;

[0023] The pitch sequence is updated based on the target pitch data to obtain the perturbed speech data.

[0024] In some embodiments, the decoupling process of the sample timbre vector, the sample content vector, and the sample object feature vector to obtain decoupling loss data includes:

[0025] The correlation between the sample content vector and the sample object feature vector is calculated to obtain the first correlation data.

[0026] The sample content vector and the sample object feature vector are subjected to feature fusion processing to obtain the speech fusion vector;

[0027] The correlation between the speech fusion vector and the sample timbre vector is calculated to obtain the second correlation data.

[0028] The decoupling loss data is obtained based on the first relevant data and the second relevant data; wherein, the first relevant data is positively correlated with the decoupling loss data, and the second relevant data is negatively correlated with the decoupling loss data.

[0029] In some embodiments, the sample object feature vector includes a sample rhythm vector and a sample pitch vector;

[0030] The step of calculating the correlation between the sample content vector and the sample object feature vector to obtain the first relevant data includes:

[0031] The correlation between the sample rhythm vector and the sample pitch vector is calculated to obtain the first sub-correlation data;

[0032] The correlation between the sample rhythm vector and the sample content vector is calculated to obtain the second sub-correlation data.

[0033] The correlation between the sample pitch vector and the sample content vector is calculated to obtain the third sub-correlation data;

[0034] The first sub-related data, the second sub-related data, and the third sub-related data are summed to obtain the first related data.

[0035] In some embodiments, the original object feature encoder includes an original rhythm encoder and an original pitch encoder;

[0036] The step of performing object feature encoding processing on the sample speech data through the original object feature encoder to obtain the sample object feature vector includes:

[0037] The original rhythm encoder is used to perform rhythm encoding processing on the sample speech data to obtain the sample rhythm vector;

[0038] The original pitch encoder is used to perform pitch encoding processing on the sample speech data to obtain the sample pitch vector.

[0039] In some embodiments, the step of converting the acquired target speech data using the target speech model and preset target conversion object data to obtain converted speech data includes:

[0040] The target speech data is reconstructed by using the target speech model and preset target conversion object data to obtain target speech reconstruction data.

[0041] The target speech reconstruction data is processed by a preset vocoder to obtain the converted speech data.

[0042] To achieve the above objectives, a second aspect of this application provides a speech conversion device, the device comprising:

[0043] The data acquisition module is used to acquire sample speech data;

[0044] The data input module is used to input the sample speech data into a preset original speech model; wherein, the original speech model includes an original timbre encoder, an original content encoder, an original object feature encoder, and an original decoder;

[0045] The timbre encoding module is used to perform timbre encoding processing on the sample speech data through the original timbre encoder to obtain a sample timbre vector;

[0046] The content encoding module is used to perform content encoding processing on the sample speech data through the original content encoder to obtain a sample content vector.

[0047] The object feature encoding module is used to perform object feature encoding processing on the sample speech data through the original object feature encoder to obtain the sample object feature vector;

[0048] The speech reconstruction module is used to perform speech reconstruction processing on the sample timbre vector, the sample content vector, and the sample object feature vector through the original decoder to obtain sample speech reconstruction data.

[0049] The speech comparison module is used to perform speech comparison based on the sample speech data and the sample speech reconstruction data to obtain speech loss data.

[0050] The decoupling module is used to decouple the sample timbre vector, the sample content vector, and the sample object feature vector to obtain decoupling loss data.

[0051] The parameter adjustment module is used to adjust the parameters of the original speech model according to the speech loss data and the decoupling loss data to obtain the target speech model.

[0052] The speech conversion module is used to convert the acquired target speech data into speech data using the target speech model and preset target conversion object data.

[0053] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0054] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0055] The speech conversion method, apparatus, electronic device, and storage medium proposed in this application encode sample speech data using an original speech model to obtain timbre vectors, sample content vectors, and sample object feature vectors. Speech reconstruction processing is then performed on the timbre vectors, sample content vectors, and sample object feature vectors to obtain sample speech reconstruction data. Speech loss data is obtained from the sample speech reconstruction data and sample speech data, and decoupling loss data is obtained from the timbre vectors, sample content vectors, and sample object feature vectors. Since the speech loss data characterizes the speech differences between the sample speech reconstruction data and the sample speech data, and the decoupling loss data characterizes the coupling degree between the timbre vectors, sample content vectors, and sample object feature vectors, when the speech loss data and decoupling loss data are used together as loss data to adjust the parameters of the original speech model, a target speech model with more accurate feature extraction and lower coupling between the extracted vectors can be obtained. Therefore, when speech conversion is performed based on the target speech model, the problem of speech conversion accuracy being affected by the correlation between speech representation vectors and speech content in related technologies can be avoided; that is, the embodiments of this application can improve the accuracy of speech conversion. When this application is applied to scenarios such as intelligent diagnosis and treatment, remote consultation, and online consultation in digital healthcare, it can improve the accuracy of speech conversion of the person being questioned, and protect the privacy of the person being questioned while minimizing the impact on the content of their questions. Attached Figure Description

[0056] Figure 1 This is a flowchart of the speech conversion method provided in the embodiments of this application;

[0057] Figure 2 yes Figure 1 The flowchart of step S104 in the process;

[0058] Figure 3 yes Figure 2 The flowchart of step S201 in the text;

[0059] Figure 4 yes Figure 1 The flowchart of step S105 in the process;

[0060] Figure 5 yes Figure 1 The flowchart of step S108 in the process;

[0061] Figure 6yes Figure 5 The flowchart of step S501 in the text;

[0062] Figure 7 yes Figure 1 The flowchart of step S110 in the middle;

[0063] Figure 8 This is a schematic diagram of the structure of the speech conversion device provided in the embodiments of this application;

[0064] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0068] First, let's analyze some of the terms used in this application:

[0069] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0070] Mutual information is a measure of the interdependence between two random variables, denoted by I(X,Y). Mutual information measures the information shared by two random variables, that is, the degree to which the uncertainty of random variable Y is reduced when determining random variable X; or the degree to which the uncertainty of random variable X is reduced when determining random variable Y. Specifically, mutual information can be calculated according to the following equation (1).

[0071]

[0072] Where p(X) represents the marginal distribution of vector X, p(Y) represents the marginal distribution of vector Y, and p(X,Y) represents the joint distribution of vectors X and Y.

[0073] Pre-emphasis is a preprocessing technique used in feature extraction. Since the energy of high-frequency signals in speech signals is typically low and subject to significant suppression, it is necessary to increase the energy of the high-frequency components, i.e., compensate for the amplitude of the high-frequency components of the speech signal. This makes the energy distribution of the speech signal more balanced, thereby preventing instability in the numerical calculation of the Fourier transform.

[0074] Framing is a preprocessing method in feature extraction. Speech signals are non-stationary, but considering the regular rhythmic movement of the vocal cords during voiced speech, meaning the fundamental frequency is relatively fixed within a short time range, speech signals exhibit short-time stationarity. Specifically, the speech signal within 10ms to 30ms is considered a quasi-stationary process. Short-time analysis primarily uses framing, but the fundamental frequency may change between adjacent frames. To ensure the smoothness of acoustic feature parameters, overlapping frames are generally used. Specifically, the speech signal is segmented while maintaining a certain repetition rate. Framing is equivalent to adding a rectangular window to the speech signal; therefore, the main parameters of framing include frame length (the length of each frame) and frame shift (the distance the rectangular window moves).

[0075] Windowing is a preprocessing technique used in feature extraction. Considering the short-term stationarity of speech signals, windowing is applied to each frame of the signal. Common window functions include the Hamming window, Hanning window, and Blackman window. Specifically, after dividing the speech signal into frames, each frame is substituted into the window function, with the values ​​outside the window set to 0 to eliminate potential signal discontinuities at the ends of each frame. For example, a Hamming window of size n is applied to each m frames of the signal. By applying a Hamming window to each frame of data, the Hamming window matrix C(m, n) is obtained.

[0076] Fourier Transform: This refers to the ability to represent a function satisfying certain conditions as a trigonometric function (sine or cosine function) or a linear combination of their integrals. In feature extraction, a Fast Fourier Transform (FFT) of N = 4*n is performed on each frame, resulting in a matrix of size (m, 4n). The number of frames after the Fourier Transform is still m. The modulus of the distribution of the 4n data points in each frame is taken, squared, and then divided by 4n to obtain the energy spectral density E(m, 4n), which has a size of m*4n. Then, the energy obtained from each frame is summed to obtain an m*1 matrix, where each element represents the sum of the energy in one frame.

[0077] Mel filter bank: Defines a filter bank with M filters, where the number of filters is similar to the number of critical bands, and M is typically 22-26. The energy spectrum of a speech signal is processed using a set of triangular filters with a Mel scale, constructed as described above, to extract features from the speech signal. The triangular filters have the following functions: First, they are dense at low frequencies and sparse at high frequencies, thus mimicking the high resolution of low-frequency signals by the human ear. Second, they smooth the spectrum and eliminate harmonics, thereby highlighting the formants of the original speech signal. It can be understood that the spectrum has an envelope and fine structure, corresponding to timbre and pitch, respectively. For speech signals, timbre is the primary useful information; integration within the triangle can eliminate the fine structure, retaining only timbre information. Third, the sequence obtained from the Fourier transform is relatively long; the triangular filters transform it into the energy under each triangle, thereby reducing the amount of data.

[0078] Speech conversion refers to converting the speech of a source speaker into the speech of a target speaker without altering the content of the source speaker's speech. For example, in digital healthcare scenarios, during intelligent diagnosis, remote consultations, and online consultations, speech conversion can be performed on the speech of the person being consulted to protect their privacy, such as converting the speech of the person being consulted (Y) into the speech of the person being consulted (Y). In related technologies, the speech of the source speaker (i.e., the person being consulted) includes speech content and different speech representation vectors. The correlation between these different speech representation vectors and speech content leads to low accuracy when directly converting speech based on the speech characteristics of the target speaker (i.e., Y). Therefore, improving the accuracy of speech conversion has become an urgent technical problem to be solved.

[0079] Based on this, embodiments of this application provide a speech conversion method and apparatus, an electronic device and a storage medium, aiming to improve the accuracy of speech conversion.

[0080] The speech conversion method, apparatus, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the speech conversion method in this application is described.

[0081] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0082] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0083] The speech conversion method provided in this application relates to the field of digital medical technology. The speech conversion method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the speech conversion method, but is not limited to the above forms.

[0084] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0085] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user voice data, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0086] Figure 1 This is an optional flowchart of the speech conversion method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S110.

[0087] Step S101: Obtain sample speech data;

[0088] Step S102: Input the sample speech data into the preset original speech model; wherein, the original speech model includes an original timbre encoder, an original content encoder, an original object feature encoder, and an original decoder;

[0089] Step S103: Perform timbre encoding on the sample speech data using the original timbre encoder to obtain the sample timbre vector;

[0090] Step S104: Perform content encoding processing on the sample speech data using the original content encoder to obtain the sample content vector;

[0091] Step S105: Perform object feature encoding on the sample speech data using the original object feature encoder to obtain the sample object feature vector;

[0092] Step S106: Perform speech reconstruction processing on the sample timbre vector, sample content vector, and sample object feature vector using the original decoder to obtain sample speech reconstruction data;

[0093] Step S107: Perform speech comparison based on sample speech data and sample speech reconstruction data to obtain speech loss data;

[0094] Step S108: Decouple the sample timbre vector, sample content vector, and sample object feature vector to obtain decoupling loss data;

[0095] Step S109: Adjust the parameters of the original speech model based on the speech loss data and decoupling loss data to obtain the target speech model;

[0096] Step S110: The acquired target speech data is converted into speech data by using the target speech model and preset target conversion object data.

[0097] Steps S101 to S110 of this embodiment involve encoding the sample speech data using the original speech model to obtain timbre vectors, sample content vectors, and sample object feature vectors. Speech reconstruction processing is then performed on the timbre vectors, sample content vectors, and sample object feature vectors to obtain sample speech reconstruction data. Speech loss data is obtained from the sample speech reconstruction data and the sample speech data, and decoupling loss data is obtained from the timbre vectors, sample content vectors, and sample object feature vectors. Since the speech loss data characterizes the speech differences between the sample speech reconstruction data and the sample speech data, and the decoupling loss data characterizes the coupling degree between the timbre vectors, sample content vectors, and sample object feature vectors, when the speech loss data and decoupling loss data are used together as loss data to adjust the parameters of the original speech model, a target speech model with more accurate feature extraction and lower coupling between the extracted vectors can be obtained. Therefore, when speech conversion is performed based on the target speech model, the problem of low speech conversion accuracy caused by the correlation between speech representation vectors and speech content in related technologies can be avoided. In other words, this embodiment can improve the accuracy of speech conversion. When this application is applied to scenarios such as intelligent diagnosis and treatment, remote consultation, and online consultation in digital healthcare, it can convert the voice of the person being questioned into a speech, improving the accuracy of the speech conversion without changing the content of the question, and thus protecting the privacy of the person being questioned.

[0098] In step S101 of some embodiments, sample voice data of the sample object is obtained through methods such as Application Programming Interface (API). For example, when this application is applied to a terminal, the terminal may have applications with voice input functions installed, such as instant messaging applications, voice input method applications, voice assistants, etc. Sample voice data of the sample object is obtained according to the API interface of the aforementioned application. In addition, audio extraction can be performed on video data and audio data obtained through big data to obtain corresponding sample voice data. When this application is applied to a digital medical scenario, the sample voice data may be the voice data of the person being questioned, such as voice data describing symptoms. Therefore, it can be seen that the embodiments of this application do not specifically limit the method of obtaining sample voice data.

[0099] In step S102 of some embodiments, an original speech model is pre-set, including an original timbre encoder, an original content encoder, an original object feature encoder, and an original decoder. The input data for the original timbre encoder, the original content encoder, and the original object feature encoder are all sample speech data, and the input data for the original decoder is the output data of the original timbre encoder, the original content encoder, and the original object feature encoder. That is, the original speech model is a model with an encoder-decoder structure.

[0100] In step S103 of some embodiments, the sample speech data is used as input data to the original timbre encoder, so that the timbre features in the sample speech data are extracted and encoded by the original timbre encoder to obtain the sample timbre vector. It can be understood that timbre features are features that can reflect the speech characteristics of the sample object, and the timbre features are related to the identity of the sample object, that is, the speech data corresponding to sample object A and sample object B with different identities have different timbre features.

[0101] In step S104 of some embodiments, the sample speech data is used as input data to the original content encoder, so that the original content encoder can extract and encode the content information features to be expressed by the sample speech data to obtain a sample content vector. For example, if the content information to be expressed by the sample speech data is "I woke up this morning with a headache and weakness in my limbs", then the sample content vector is a feature vector corresponding to the above content information.

[0102] Reference Figure 2 In some embodiments, step S104 includes, but is not limited to, steps S201 to S202.

[0103] Step S201: Perform speech perturbation processing on the sample speech data to obtain perturbed speech data;

[0104] Step S202: Perform content encoding processing on the perturbed speech data using the original content encoder to obtain the sample content vector.

[0105] In step S201 of some embodiments, in order to improve the capability of the original content encoder, even if the original content encoder extracts an accurate sample content vector from the sample speech data, speech perturbation processing is performed on the sample speech data to obtain perturbed speech data. Specifically, the speech perturbation processing operation refers to the operation used to change the prosodic features in the sample speech data. Among them, prosodic features include pitch features, rhythm features, etc.

[0106] Reference Figure 3 In some embodiments, step S201 includes, but is not limited to, steps S301 to S305.

[0107] Step S301: Extract pitch from the sample speech data to obtain the original pitch data;

[0108] Step S302: Sort the original pitch data to obtain a pitch sequence;

[0109] Step S303: Extract median features from the pitch sequence to obtain candidate pitch data;

[0110] Step S304: Adjust the pitch of the candidate pitch data to obtain the target pitch data;

[0111] Step S305: Update the pitch sequence according to the target pitch data to obtain perturbed speech data.

[0112] In step S301 of some embodiments, the pitch of the sample speech data is extracted through the API interface of extraction tools such as Aubio, Essentia, Librosa, ParseLmouth, or other methods to obtain the original pitch data.

[0113] In some embodiments, steps S302 to S303 involve sorting the extracted raw pitch data by pitch values ​​to obtain a pitch sequence. The median of the pitch values ​​in the pitch sequence is then obtained to acquire candidate pitch data.

[0114] In step S304 of some embodiments, the candidate pitch data is scaled and adjusted according to a preset pitch ratio, thereby adjusting the pitch of the candidate pitch data to obtain the target pitch data.

[0115] In step S305 of some embodiments, since the candidate pitch data is used to characterize the median level of the overall pitch values ​​in the pitch sequence, when the target pitch data is used as the new median pitch value of the pitch sequence, the other original pitch data in the pitch sequence will be adaptively adjusted, thereby achieving the adjustment of the pitch sequence. Based on the adjusted pitch sequence, perturbed speech data with pitches different from the sample speech data is obtained.

[0116] In step S202 of some embodiments, the perturbed speech data is used as input data to the original content encoder, so that the original content encoder can extract content features from the perturbed speech data to obtain a sample content vector. During the training of the original speech model, the original content encoder continuously updates its ability to mine real content features from the perturbed speech data, that is, updates its ability to avoid interference from perturbed features, thereby improving the accuracy of the original content encoder in extracting content features.

[0117] In step S105 of some embodiments, the sample speech data is used as input data to the original object feature encoder. The original object feature encoder extracts and encodes speech features (excluding timbre) from the sample speech data that characterize the sample object, obtaining a sample object feature vector. For example, the original object feature encoder is used to extract pitch features, rhythm features, etc., from the sample speech data. Pitch features are the intonation features of the sample object in the sample speech data, such as the feature that the sample object expresses a certain syllable with a rising or falling intonation. It is understood that pitch features also characterize the gender information of the sample object. For example, a sample object whose gender information is female typically has a higher range of pitch features, while a sample object whose gender information is male typically has a lower range of pitch features.

[0118] Reference Figure 4 In some embodiments, the original object feature encoder includes an original syllable encoder and an original pitch encoder, and step S105 includes, but is not limited to, steps S401 to S402.

[0119] Step S401: Perform rhythm encoding processing on the sample speech data using the original rhythm encoder to obtain the sample rhythm vector;

[0120] Step S402: Perform pitch encoding processing on the sample speech data using the original pitch encoder to obtain the sample pitch vector.

[0121] In step S401 of some embodiments, the sample speech data is used as input data of the original rhythm encoder. The rhythm features of the sample speech data are extracted and encoded by the original rhythm encoder to obtain the sample rhythm vector.

[0122] In step S402 of some embodiments, the sample speech data is used as input data of the original pitch encoder. The pitch features of the sample speech data are extracted and encoded by the original pitch encoder to obtain the sample rhythm vector.

[0123] In step S106 of some embodiments, the sample timbre vector, sample content vector, and sample object feature vector are used as input data to the original decoder. The original decoder performs speech reconstruction processing on the sample timbre vector, sample content vector, and sample object feature vector, that is, generates new speech data based on the above vectors to obtain sample speech reconstruction data. It can be understood that, ideally, the content information to be expressed by the sample semantic reconstruction data should be the same as that of the sample semantic data, that is, "I woke up this morning with a headache and weakness in my limbs".

[0124] In step S107 of some embodiments, loss calculation is performed on the sample speech data and the sample speech reconstruction data according to a preset loss function to obtain speech loss data that characterizes the difference between the input data and output data of the original speech model. Therefore, when the parameters of the original speech model are adjusted according to the speech loss data, the difference between the output data and the input data of the original speech model can be reduced, thereby improving the accuracy of feature extraction by the original speech model. It is understood that the preset loss function can be any of the mean square error loss function (MSE-loss), mean absolute error (L1-loss), etc., and this application embodiment does not specifically limit it.

[0125] Understandably, when the sample speech reconstruction data is represented in Mel spectrum form, the sample speech data used to calculate speech loss data should also be represented in Mel spectrum form. Therefore, before inputting the sample speech data into the original speech model, it is necessary to convert the sample speech data into Mel spectrum form so that the speech loss data of the original speech model can be determined based on the two Mel spectra mentioned above. Specifically, the Mel spectrum corresponding to the sample speech data can be obtained by processing the sample speech data through pre-emphasis, framing, windowing, Fourier transform, and Mel filter bank.

[0126] In step S108 of some embodiments, due to the correlation between the sample timbre vector, sample content vector, and sample object feature vector, the sample timbre vector and sample object feature vector may contain feature vectors associated with the sample content vector. Since speech conversion essentially involves replacing the timbre vector, directly replacing the timbre vector extracted from the original speech model with the target timbre vector will lead to inaccurate content expression and low naturalness in the resulting speech data. Therefore, it is necessary to decouple the sample timbre vector, sample content vector, and sample object feature vector, i.e., to unwrap them. It is understood that the above description of the essence of speech conversion is only a basic description; depending on the actual conversion requirements, vectors other than the content vector can also be replaced, and this application does not specifically limit this.

[0127] It is understandable that mutual information can be used to reflect the degree of dependency between vectors. Therefore, the effect of decoupling processing can be determined by the mutual information between sample timbre vectors, sample content vectors, and sample object feature vectors. In this embodiment, the mutual information calculated based on the sample timbre vectors, sample content vectors, and sample object feature vectors is used as the model loss data of the original speech model, i.e., the decoupling loss data. This allows the decoupling loss data to continuously decrease during iterative training of the original speech model, based on the characteristic that iterative training continuously reduces the model loss data. This reduces the degree of dependency between the sample timbre vectors, sample content vectors, and sample object feature vectors, i.e., reduces the correlation between these vectors. Therefore, when speech conversion is performed based on the data output by the trained original speech model, the accuracy of the converted speech content and the naturalness of the converted speech can be improved.

[0128] Reference Figure 5 In some embodiments, step S108 includes, but is not limited to, steps S501 to S504.

[0129] Step S501: Calculate the correlation between the sample content vector and the sample object feature vector to obtain the first correlation data;

[0130] Step S502: Perform feature fusion processing on the sample content vector and the sample object feature vector to obtain the speech fusion vector;

[0131] Step S503: Calculate the correlation between the speech fusion vector and the sample timbre vector to obtain the second correlation data;

[0132] Step S504: Obtain decoupling loss data based on the first correlation data and the second correlation data; wherein, the first correlation data is the positive correlation data of the decoupling loss data, and the second correlation data is the negative correlation data of the decoupling loss data.

[0133] In step S501 of some embodiments, the correlation between the encoded sample content vector and the sample object feature vector is calculated to determine the correlation between the sample content vector and the sample object feature vector, thereby obtaining first correlation data. It is understood that since mutual information can be used to reflect the degree of dependence between vectors, the mutual information value between the sample content vector and the sample object feature vector can be calculated and used as the first correlation data I1.

[0134] Reference Figure 6 In some embodiments, the sample object feature vector includes a sample rhythm vector and a sample pitch vector. Step S501 includes, but is not limited to, steps S601 to S604.

[0135] Step S601: Calculate the correlation between the sample rhythm vector and the sample pitch vector to obtain the first sub-correlation data;

[0136] Step S602: Calculate the correlation between the sample rhythm vector and the sample content vector to obtain the second sub-correlation data;

[0137] Step S603: Calculate the correlation between the sample pitch vector and the sample content vector to obtain the third sub-correlation data;

[0138] Step S604: Sum the first sub-correlated data, the second sub-correlated data, and the third sub-correlated data to obtain the first related data.

[0139] In step S601 of some embodiments, the sample rhythm vector Z is calculated. r With the sample pitch vector Z p The mutual information between the samples is used to determine the correlation between the sample rhythm vector and the sample pitch vector, and to obtain the corresponding first sub-correlation data I(Z). r Z p ).

[0140] In step S602 of some embodiments, the sample rhythm vector Z is calculated. r With the sample content vector Z c The mutual information between the sample rhythm vector and the sample content vector is used to determine the correlation between them, and to obtain the corresponding second sub-correlation data I(Z). r Z c ).

[0141] In step S603 of some embodiments, the sample pitch vector Z is calculated.p With the sample content vector Z c The mutual information between them is used to determine the correlation between the pitch vector and the sample content vector, and to obtain the corresponding third sub-correlation data I(Z). p Z c ).

[0142] In step S604 of some embodiments, the first relevant data I1 is calculated according to the following formula (2).

[0143] I1=I(Z r Z p )+I(Z r Z c )+I(Z p Z c Equation (2)

[0144] In steps S502 to S503 of some embodiments, the sample content vector and the sample object feature vector are subjected to feature fusion processing, that is, the sample content vector and the sample object feature vector are treated as a whole vector to obtain a speech fusion vector. The correlation between the speech fusion vector obtained by feature fusion and the sample timbre vector is calculated to determine the correlation between the speech fusion vector and the sample timbre vector, thus obtaining second correlation data. It is understood that since mutual information can be used to reflect the degree of dependence between vectors, the mutual information between the speech fusion vector and the sample timbre vector can be calculated, and the mutual information value can be used as the second correlation data I2.

[0145] In some embodiments, when the sample object feature vector includes a sample rhythm vector and a sample pitch vector, the speech fusion vector is {Z}. c Z r Z p Therefore, the second relevant data I2 = I({Z) c Z r Z p}, Z t ), where Z t The timbre features of the sample.

[0146] In step S504 of some embodiments, the decoupling loss data L is calculated according to the following formula (3). MI .

[0147] L MI =I1-I2......Equation (3)

[0148] In this process, the decoupling loss data is used as the model loss data to adjust the parameters of the original speech model during training. Based on the characteristic that iterative training continuously reduces the model loss data, the decoupling loss data will also continuously decrease during iterative training. Furthermore, to ensure the accuracy of the vectors extracted from the original speech model while achieving sufficient decoupling between the extracted vectors, the first relevant data I1 needs to be minimized, and the second relevant data I2 needs to be maximized. Therefore, the first relevant data I1 is used as the decoupling loss data L. MI Positively correlated data, so that the numerical trend of the first correlated data I1 is related to the decoupling loss data L. MI Similarly, even though the value of the first relevant data I1 continuously decreases during iterative training, the second relevant data I2 is used as the decoupling loss data L. MI The negative correlation data, so that the numerical trend of the second correlation data I2 is related to the decoupling loss data L. MI Conversely, the value of the second relevant data I2 continuously increases during iterative training. It is understood that the embodiments of this application indirectly constrain the size of mutual information by minimizing the upper bound of mutual information and maximizing the lower bound of mutual information.

[0149] In step S109 of some embodiments, the parameters of the original speech model are adjusted based on the speech loss data and decoupling loss data to obtain a target speech model with stronger feature extraction capabilities and lower output data correlation. Specifically, the speech loss data and decoupling loss data can be directly summed, and the parameters of the original speech model can be adjusted based on the data obtained from the summation. Alternatively, different weights can be set for the speech loss data and decoupling loss data, and the parameters of the original speech model can be adjusted based on the data obtained from the weighted summation. This application does not specifically limit the specific implementation of these methods.

[0150] In step S110 of some embodiments, target speech data of the target object is acquired, features are extracted from the target speech data according to the target speech model, and some of the extracted features are replaced with target conversion object data. The target conversion object data is the speech feature data of the target conversion object, such as timbre feature data. Therefore, according to the above operations, target speech data can be converted into speech data with the speech feature data of the target conversion object, i.e., speech data is converted, thereby achieving speech conversion. Furthermore, target speech data of the target object can be acquired through API interfaces or other means. For example, when this application is applied to a digital healthcare scenario, target speech data can be acquired through intelligent diagnosis and treatment API interfaces, remote consultation API interfaces, online consultation API interfaces, etc. When this application is applied to a terminal, the terminal can install applications with voice input functions, such as instant messaging applications, voice input method applications, voice assistants, etc. The target object can use these applications to input the target speech data to be converted. It should be noted that in practical applications, when the target object uses an instant messaging application for speech conversion, it can click on the voice conversion interface of the instant messaging application and input the target speech data through the voice input control set on the interface. At this point, the instant messaging application obtains the target voice data by calling a voice acquisition device, such as turning on the microphone.

[0151] It is understandable that the target conversion object data can also be identity identification data. A pitch database is pre-set, which includes timbre feature data that has a mapping relationship with the identity identification data. When the target object uses an instant messaging application for voice conversion, or performs voice conversion using a pre-set voice conversion interface in a digital medical scenario, the voice conversion interface can also be set with identity identification controls corresponding to different identity identification data. By obtaining the instruction from the target object to trigger the generation of the identity identification control, the corresponding timbre feature data is obtained from the pitch database according to the instruction, thereby realizing the voice conversion of the target voice data based on the timbre feature data. For example, controls for "female voice", "male voice", "child's voice", "elderly voice", etc. are set. If the target object is female, when the instruction from the target object to trigger the "male voice" control is obtained, the timbre of the target voice data can be converted from the female timbre corresponding to the target object to the male timbre without changing the content of the symptoms described by the target object. Therefore, it can be seen that the embodiments of this application do not specifically limit the data type and acquisition method of the target object data.

[0152] Reference Figure 7 In some embodiments, step S110 includes, but is not limited to, steps S701 to S702.

[0153] Step S701: Perform speech reconstruction processing on the target speech data using the target speech model and preset target conversion object data to obtain target speech reconstruction data;

[0154] Step S702: The target speech reconstruction data is processed by a preset vocoder to obtain converted speech data.

[0155] In step S701 of some embodiments, the target speech model includes a target content encoder, a target object feature encoder, and a target decoder. The target speech data is used as input data for the target content encoder and the target object feature encoder, respectively, to obtain corresponding target content vectors and target object feature vectors. The target content vector, the target object feature vector, and preset target conversion object data are used as input data for the target decoder. The target decoder performs speech reconstruction processing on the target speech data and the target conversion object data to obtain target speech reconstruction data with the timbre feature data corresponding to the target conversion object data as the target timbre vector.

[0156] In some embodiments, in step S702, since the data output by the target decoder is in the form of Mel spectrum, the target speech reconstruction data also needs to be input to a preset vocoder to output audio data corresponding to the target speech reconstruction data, thereby realizing the speech conversion of the target speech data.

[0157] The speech conversion method provided in this application encodes sample speech data using an original speech model to obtain timbre vectors, sample content vectors, and sample object feature vectors. Speech reconstruction is then performed on the timbre vectors, sample content vectors, and sample object feature vectors to obtain sample speech reconstruction data. Speech loss data is obtained from the sample speech reconstruction data and the sample speech data, and decoupling loss data is obtained from the timbre vectors, sample content vectors, and sample object feature vectors. Since the speech loss data characterizes the speech differences between the sample speech reconstruction data and the sample speech data, and the decoupling loss data characterizes the coupling degree between the timbre vectors, sample content vectors, and sample object feature vectors, when the speech loss data and decoupling loss data are used together as loss data to adjust the parameters of the original speech model, a target speech model with more accurate feature extraction and lower coupling between the extracted vectors can be obtained. Therefore, when speech conversion is performed based on the target speech model, the problem of speech conversion accuracy caused by the correlation between speech representation vectors and speech content in related technologies can be avoided; that is, the embodiments of this application can improve the accuracy of speech conversion. When this application is applied to scenarios such as intelligent diagnosis and treatment, remote consultation, and online consultation in digital healthcare, it can improve the accuracy of speech conversion of the person being questioned, and protect the privacy of the person being questioned while minimizing the impact on the content of their questions.

[0158] Please see Figure 8 This application also provides a speech conversion device that can implement the above-described speech conversion method. The device includes:

[0159] Data acquisition module 801 is used to acquire sample speech data;

[0160] The data input module 802 is used to input sample speech data into a preset original speech model; wherein, the original speech model includes an original timbre encoder, an original content encoder, an original object feature encoder, and an original decoder;

[0161] The timbre encoding module 803 is used to perform timbre encoding processing on the sample speech data through the original timbre encoder to obtain the sample timbre vector;

[0162] The content encoding module 804 is used to perform content encoding processing on the sample speech data through the original content encoder to obtain the sample content vector.

[0163] The object feature encoding module 805 is used to perform object feature encoding on the sample speech data through the original object feature encoder to obtain the sample object feature vector.

[0164] The speech reconstruction module 806 is used to perform speech reconstruction processing on the sample timbre vector, sample content vector and sample object feature vector through the original decoder to obtain sample speech reconstruction data.

[0165] The speech comparison module 807 is used to perform speech comparison based on sample speech data and sample speech reconstruction data to obtain speech loss data.

[0166] The decoupling module 808 is used to decouple the sample timbre vector, sample content vector, and sample object feature vector to obtain decoupling loss data.

[0167] The parameter adjustment module 809 is used to adjust the parameters of the original speech model based on the speech loss data and decoupling loss data to obtain the target speech model.

[0168] The speech conversion module 810 is used to convert the acquired target speech data into speech data by using the target speech model and preset target conversion object data.

[0169] The specific implementation of this voice conversion device is basically the same as the specific implementation of the above-described voice conversion method, and will not be repeated here.

[0170] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described voice conversion method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0171] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0172] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0173] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and called and executed by the processor 901 using the speech conversion method of the embodiments of this application.

[0174] The input / output interface 903 is used to implement information input and output;

[0175] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0176] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0177] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0178] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described speech conversion method.

[0179] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0180] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0181] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0182] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0184] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0185] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0187] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0189] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0190] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A speech conversion method, characterized in that, The method includes: Acquire sample speech data; The sample speech data is input into a preset original speech model; wherein, the original speech model includes an original timbre encoder, an original content encoder, an original object feature encoder, and an original decoder; The original timbre encoder is used to perform timbre encoding on the sample speech data to obtain a sample timbre vector. The original content encoder performs content encoding on the sample speech data to obtain a sample content vector; The original object feature encoder is used to perform object feature encoding on the sample speech data to obtain the sample object feature vector; The original decoder performs speech reconstruction processing on the sample timbre vector, the sample content vector, and the sample object feature vector to obtain sample speech reconstruction data. Speech comparison is performed based on the sample speech data and the sample speech reconstruction data to obtain speech loss data; The sample timbre vector, the sample content vector, and the sample object feature vector are decoupled to obtain decoupling loss data. The original speech model is adjusted according to the speech loss data and the decoupling loss data to obtain the target speech model. The acquired target speech data is converted into speech data by using the target speech model and preset target conversion object data.

2. The method according to claim 1, characterized in that, The step of performing content encoding processing on the sample speech data through the original content encoder to obtain a sample content vector includes: The sample speech data is subjected to speech perturbation processing to obtain perturbed speech data; The perturbation speech data is processed by the original content encoder to obtain the sample content vector.

3. The method according to claim 2, characterized in that, The step of performing speech perturbation processing on the sample speech data to obtain perturbed speech data includes: The pitch of the sample speech data is extracted to obtain the original pitch data; The original pitch data is sorted to obtain a pitch sequence; Median feature extraction is performed on the pitch sequence to obtain candidate pitch data; The candidate pitch data is adjusted to obtain the target pitch data; The pitch sequence is updated based on the target pitch data to obtain the perturbed speech data.

4. The method according to claim 1, characterized in that, The process of decoupling the sample timbre vector, the sample content vector, and the sample object feature vector to obtain decoupling loss data includes: The correlation between the sample content vector and the sample object feature vector is calculated to obtain the first correlation data. The sample content vector and the sample object feature vector are subjected to feature fusion processing to obtain the speech fusion vector; The correlation between the speech fusion vector and the sample timbre vector is calculated to obtain the second correlation data. The decoupling loss data is obtained based on the first relevant data and the second relevant data; wherein, the first relevant data is positively correlated with the decoupling loss data, and the second relevant data is negatively correlated with the decoupling loss data.

5. The method according to claim 4, characterized in that, The feature vector of the sample object includes the sample rhythm vector and the sample pitch vector; The step of calculating the correlation between the sample content vector and the sample object feature vector to obtain the first relevant data includes: The correlation between the sample rhythm vector and the sample pitch vector is calculated to obtain the first sub-correlation data; The correlation between the sample rhythm vector and the sample content vector is calculated to obtain the second sub-correlation data. The correlation between the sample pitch vector and the sample content vector is calculated to obtain the third sub-correlation data; The first sub-related data, the second sub-related data, and the third sub-related data are summed to obtain the first related data.

6. The method according to any one of claims 1 to 5, characterized in that, The original object feature encoder includes an original rhythm encoder and an original pitch encoder; The step of performing object feature encoding processing on the sample speech data through the original object feature encoder to obtain the sample object feature vector includes: The original rhythm encoder is used to perform rhythm encoding processing on the sample speech data to obtain the sample rhythm vector; The original pitch encoder is used to perform pitch encoding processing on the sample speech data to obtain the sample pitch vector.

7. The method according to claim 6, characterized in that, The step of converting the acquired target speech data using the target speech model and preset target conversion object data to obtain converted speech data includes: The target speech data is reconstructed by using the target speech model and preset target conversion object data to obtain target speech reconstruction data. The target speech reconstruction data is processed by a preset vocoder to obtain the converted speech data.

8. A voice conversion device, characterized in that, The device includes: The data acquisition module is used to acquire sample speech data; The data input module is used to input the sample speech data into a preset original speech model; wherein, the original speech model includes an original timbre encoder, an original content encoder, an original object feature encoder, and an original decoder; The timbre encoding module is used to perform timbre encoding processing on the sample speech data through the original timbre encoder to obtain a sample timbre vector; The content encoding module is used to perform content encoding processing on the sample speech data through the original content encoder to obtain a sample content vector. The object feature encoding module is used to perform object feature encoding processing on the sample speech data through the original object feature encoder to obtain the sample object feature vector; The speech reconstruction module is used to perform speech reconstruction processing on the sample timbre vector, the sample content vector, and the sample object feature vector through the original decoder to obtain sample speech reconstruction data. The speech comparison module is used to perform speech comparison based on the sample speech data and the sample speech reconstruction data to obtain speech loss data. The decoupling module is used to decouple the sample timbre vector, the sample content vector, and the sample object feature vector to obtain decoupling loss data; The parameter adjustment module is used to adjust the parameters of the original speech model according to the speech loss data and the decoupling loss data to obtain the target speech model. The speech conversion module is used to convert the acquired target speech data into speech data using the target speech model and preset target conversion object data.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images