Data processing method and device, electronic equipment and computer storage medium

Abstract

The invention provides a data processing method and device, electronic equipment and a computer storage medium, and the method comprises the steps: obtaining sample data corresponding to a target user, and the sample data comprises the voice information of the target user; training a preset initial sound cloning model based on the sample data to obtain a first target sound cloning model corresponding to the target user; when it is detected that a sound cloning instruction for the first target sound cloning model is triggered, obtaining initial voice data; and processing the initial voice data through the first target voice clone model to obtain target voice data corresponding to the target user, the tone of the target voice data being consistent with the tone of the target user. According to the embodiment provided by the scheme, the diversity of the voice data can be enhanced, and the individuation and customization level of user experience can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of sound synthesis, and in particular to a data processing method, device, electronic device and computer storage medium. Background Art

[0002] In the related art, when playing sound resources such as music and stories, a pre-set specific timbre is usually used. Such a sound resource with a fixed timbre may make the user feel monotonous when playing, and different users have different preferences for sounds, so a fixed timbre is difficult to meet the personalized needs of users. Summary of the invention

[0003] The embodiments of the present application provide a data processing method, device, electronic device and computer storage medium, which can enhance the diversity of voice data and improve the personalization and customization level of user experience. The above technical solutions are as follows:

[0004] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0005] Acquire sample data corresponding to the target user, the sample data including voice information of the target user;

[0006] Training a preset initial voice cloning model based on the sample data to obtain a first target voice cloning model corresponding to the target user;

[0007] When it is detected that the voice cloning instruction for the first target voice cloning model is triggered, initial voice data is acquired;

[0008] The initial voice data is processed by the first target voice cloning model to obtain target voice data corresponding to the target user, and the timbre of the target voice data is consistent with the timbre of the target user.

[0009] In a possible implementation, the step of obtaining sample data corresponding to the target user includes:

[0010] When it is detected that the target user triggers the sound customization instruction, a first preset resource list is displayed, where the first preset resource list includes a plurality of first resource identifiers corresponding to a plurality of first preset resources;

[0011] When detecting a determination operation of the target user on a first target resource identifier in the first preset resource list, obtaining a voice message sent by the target user on a target resource corresponding to the first target resource identifier, wherein the first target resource identifier is included in the plurality of first resource identifiers;

[0012] The sample data is determined according to the above voice information.

[0013] In a possible implementation, the obtaining of the voice information sent by the target user to the target resource corresponding to the first target resource identifier includes:

[0014] Displaying a content page corresponding to the first target resource identifier, wherein the content page corresponding to the first target resource identifier includes a recording button and text content of the target resource corresponding to the first target resource identifier;

[0015] When a trigger operation on the recording button in the content page is detected, voice information sent by the target user to the target resource corresponding to the first target resource identifier is collected.

[0016] In a possible implementation, the determining of sample data according to the voice information includes:

[0017] Determine the total amount of historical voice information corresponding to the target user based on the voice information, where the amount of historical voice information corresponding to the target user is the amount of voice information sent by the target user for the multiple first preset resources that has been currently acquired;

[0018] It is determined whether the total number reaches a preset threshold. If so, the historical voice information is preprocessed and the preprocessed historical voice information is determined as sample data.

[0019] In a possible implementation, the training of a preset initial voice cloning model based on the sample data to obtain a first target voice cloning model corresponding to the target user includes:

[0020] Encoding the sample data by a sound encoder in a preset initial sound cloning model to obtain a first acoustic feature corresponding to the sample data;

[0021] Fine-tuning the speech synthesis model in the initial sound cloning model based on the first acoustic feature to obtain a fine-tuned speech synthesis model;

[0022] The trained initial voice cloning model is determined as the first target voice cloning model corresponding to the target user.

[0023] In a possible implementation, the processing of the initial voice data by the first target voice cloning model to obtain the target voice data corresponding to the target user includes:

[0024] Inputting the initial voice data into the first target sound cloning model, encoding the initial voice data through a sound encoder in the first target sound cloning model, and obtaining a second acoustic feature corresponding to the initial voice data;

[0025] The initial voice data is processed by the fine-tuned voice synthesis model in the first target voice cloning model to obtain the target voice data corresponding to the target user.

[0026] In a possible implementation, the method further includes:

[0027] Displaying a second preset resource list, wherein the second preset resource list includes a plurality of second resource identifiers;

[0028] When a determination operation is detected for a second target resource identifier in the second preset resource list, a plurality of model identifiers corresponding to the second target resource identifier are displayed, wherein the sound cloning model corresponding to each model identifier in the plurality of model identifiers is used to convert the input voice data into voice data consistent with the timbre of the corresponding user;

[0029] When a trigger operation is detected for a model identifier corresponding to the first target sound cloning model among the multiple model identifiers, it is determined that the sound cloning instruction for the first target sound cloning model is triggered, and the voice data corresponding to the second target resource identifier is used as the initial voice data.

[0030] In a second aspect, an embodiment of the present application provides a data processing device, the device comprising:

[0031] A first acquisition module is used to acquire sample data corresponding to a target user, wherein the sample data includes voice information of the target user;

[0032] A training module, configured to train a preset initial voice cloning model based on the sample data to obtain a first target voice cloning model corresponding to the target user;

[0033] A second acquisition module is used to acquire initial voice data when it is detected that the voice cloning instruction for the first target voice cloning model is triggered;

[0034] The processing module is used to process the initial voice data through the first target voice cloning model to obtain the target voice data corresponding to the target user, and the timbre of the target voice data is consistent with the timbre of the target user.

[0035] In a possible implementation, the first acquisition module includes:

[0036] A display unit, configured to display a first preset resource list when it is detected that a target user triggers a sound customization instruction, wherein the first preset resource list includes a plurality of first resource identifiers corresponding to a plurality of first preset resources;

[0037] an acquisition unit, configured to acquire, when detecting a determination operation of the target user on a first target resource identifier in the first preset resource list, a voice message sent by the target user on a target resource corresponding to the first target resource identifier, wherein the first target resource identifier is included in the plurality of first resource identifiers;

[0038] The first determining unit is used to determine the sample data according to the above-mentioned voice information.

[0039] In a possible implementation, the acquisition unit includes:

[0040] A display subunit, used to display a content page corresponding to the first target resource identifier, wherein the content page corresponding to the first target resource identifier includes a recording button and text content of the target resource corresponding to the first target resource identifier;

[0041] The collection subunit is used to collect the voice information sent by the target user to the target resource corresponding to the first target resource identifier when a trigger operation on the recording button in the content page is detected.

[0042] In a possible implementation, the determining unit includes:

[0043] A determination subunit, configured to determine a total amount of historical voice information corresponding to the target user based on the voice information, wherein the amount of historical voice information corresponding to the target user is the amount of voice information sent by the target user for the plurality of first preset resources that has been currently acquired;

[0044] The judging subunit is used to judge whether the above total number reaches a preset threshold value. If so, the above historical voice information is pre-processed and the pre-processed historical voice information is determined as sample data.

[0045] In a possible implementation, the training module includes:

[0046] A first encoding unit, configured to encode the sample data by using a sound encoder in a preset initial sound cloning model to obtain a first acoustic feature corresponding to the sample data;

[0047] A fine-tuning unit, configured to fine-tune the speech synthesis model in the initial sound cloning model based on the first acoustic feature to obtain a fine-tuned speech synthesis model;

[0048] The second determining unit is used to determine the trained initial voice cloning model as the first target voice cloning model corresponding to the target user.

[0049] In a possible implementation, the processing module includes:

[0050] a second encoding unit, configured to input the initial voice data into the first target voice cloning model, and encode the initial voice data through a voice encoder in the first target voice cloning model to obtain a second acoustic feature corresponding to the initial voice data;

[0051] The processing unit is used to process the initial voice data through the fine-tuned speech synthesis model in the first target voice cloning model to obtain the target voice data corresponding to the target user.

[0052] In a possible implementation, the above device further includes:

[0053] A display module, used for displaying a second preset resource list, wherein the second preset resource list includes a plurality of second resource identifiers;

[0054] The display module is further configured to display a plurality of model identifiers corresponding to the second target resource identifier in the second preset resource list when a determination operation is detected for the second target resource identifier, wherein the sound cloning model corresponding to each model identifier in the plurality of model identifiers is configured to convert the input voice data into voice data consistent with the timbre of the corresponding user;

[0055] A determination module is used to determine that a sound cloning instruction for the first target sound cloning model is triggered when a trigger operation is detected for a model identifier corresponding to the first target sound cloning model among the multiple model identifiers, and to use the voice data corresponding to the second target resource identifier as the initial voice data.

[0056] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory;

[0057] The above-mentioned memory stores a computer program, and the above-mentioned computer program is suitable for being loaded by the above-mentioned processor and executing the steps of the method provided by the first aspect of the embodiment of the present application or any possible implementation method of the first aspect.

[0058] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores multiple instructions, and the instructions are suitable for being loaded by a processor and executing the steps of the method provided by the first aspect of the embodiment of the present application or any possible implementation method of the first aspect.

[0059] The embodiment of the present application obtains sample data corresponding to the target user, the sample data includes the voice information of the target user; trains a preset initial sound cloning model based on the sample data to obtain a first target sound cloning model corresponding to the target user; obtains initial voice data when it is detected that the sound cloning instruction for the first target sound cloning model is triggered; processes the initial voice data through the first target sound cloning model to obtain target voice data corresponding to the target user, and the timbre of the target voice data is consistent with the timbre of the target user. By collecting the user's voice to generate sample data and training the initial sound cloning model, an exclusive target sound cloning model corresponding to the user can be obtained; and then the target sound cloning model can be used to clone the specific initial voice data to obtain voice data consistent with the user's timbre, thereby enhancing the diversity of voice data and improving the level of personalization and customization of user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 A schematic diagram of the structure of a data processing system provided for an exemplary embodiment of the present application;

[0062] Figure 2 A flowchart of a data processing method provided for an exemplary embodiment of the present application;

[0063] Figure 3 A schematic diagram of a first page provided for an exemplary embodiment of the present application;

[0064] Figure 4 A schematic diagram of a content page provided for an exemplary embodiment of the present application;

[0065] Figure 5 A schematic diagram of a second page provided for an exemplary embodiment of the present application;

[0066] Figure 6 A specific flow chart of a data processing method provided for an exemplary embodiment of the present application;

[0067] Figure 7 A schematic diagram of the structure of a data processing device provided for an exemplary embodiment of the present application;

[0068] Figure 8A schematic structural diagram of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0070] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0071] Please refer to the following Figure 1 , which exemplarily shows a structural diagram of a data processing system provided in an embodiment of the present application. Figure 1 As shown, the system includes a terminal device 110 and a server 120. The terminal device 110 and the server 120 can be directly or indirectly connected via wired or wireless communication, which is not limited in the present application.

[0072] In some embodiments, the terminal device 110 can be a smart camera, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart appliance, etc., but is not limited thereto.

[0073] Server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), big data and artificial intelligence platforms.

[0074] The terminal device 110 may send a request to the server 120 , and the server 120 may process the request and return a response.

[0075] In some embodiments, the terminal device 110 is used to execute a data processing method, which includes the following steps: obtaining sample data corresponding to a target user, the sample data including voice information of the target user; training a preset initial sound cloning model based on the sample data to obtain a first target sound cloning model corresponding to the target user; obtaining initial voice data when it is detected that a sound cloning instruction for the first target sound cloning model is triggered; processing the initial voice data through the first target sound cloning model to obtain target voice data corresponding to the target user, the timbre of the target voice data being consistent with the timbre of the target user.

[0076] Optionally, the target user may be a person using a terminal device. The terminal device may run a related application software, and the target user may use the application software to upload the corresponding voice information.

[0077] Optionally, the sample data corresponding to the target user may include multiple voice information, each of which may be the voice of the target user in various environments (such as a quiet environment, a noisy environment) and different contexts (such as reading a text, asking a question, issuing a command, chatting, etc.). The multiple voice information may include different speech speeds and tones.

[0078] Optionally, the sample data corresponding to the target user may also include text information corresponding to the voice information, where the text information is information that records the voice information in text form.

[0079] Optionally, the first target voice cloning model is a model that is further trained and fine-tuned based on the initial voice cloning model, and the first target voice cloning model is a model that is customized and trained specifically for the voice features of the target user.

[0080] Optionally, the initial voice data may be a preset resource selected by the user, such as voice data such as voice stories and music stored locally on the terminal device or in a server, and the timbre of the initial voice data is the original timbre.

[0081] Optionally, the initial voice data may be input into the first target voice cloning model so that the timbre of the initial voice data is converted by the first target voice cloning model, and then the first target voice cloning model outputs target voice data corresponding to the target user.

[0082] In some embodiments, when the data processing system includes the above-mentioned terminal device 110 and the server 120, the above-mentioned server 120 can be used to execute a data processing method, which includes the following steps: obtaining sample data corresponding to the target user, the above-mentioned sample data including the voice information of the above-mentioned target user; training a preset initial sound cloning model based on the above-mentioned sample data to obtain a first target sound cloning model corresponding to the above-mentioned target user; when it is detected that the sound cloning instruction for the above-mentioned first target sound cloning model is triggered, obtaining initial voice data; processing the above-mentioned initial voice data through the above-mentioned first target sound cloning model to obtain the target voice data corresponding to the above-mentioned target user, and the timbre of the above-mentioned target voice data is consistent with the timbre of the above-mentioned target user.

[0083] An exemplary embodiment of the present application provides a data processing method. The data processing method can be applied to the above terminal device. For details, please refer to Figure 2 , which exemplarily shows a flow chart of a data processing method provided in an embodiment of the present application. Figure 2 As shown, the data processing method includes the following S21-S24:

[0084] S21. Acquire sample data corresponding to a target user, where the sample data includes voice information of the target user.

[0085] In some embodiments, the target user may be a person using a terminal device. The terminal device may run a related application software, and the target user may use the application software to upload the corresponding voice information.

[0086] In some embodiments, the sample data corresponding to the target user may include multiple voice information, each of which may be the voice of the target user in various environments (such as quiet environment, noisy environment) and different contexts (such as reading text, asking questions, issuing instructions, chatting, etc.). The multiple voice information may include different speech speeds and tones.

[0087] In some embodiments, the sample data corresponding to the target user may also include text information corresponding to the voice information, where the text information is information that records the voice information in text form.

[0088] S22: training a preset initial voice cloning model based on the sample data to obtain a first target voice cloning model corresponding to the target user.

[0089] In some embodiments, the initial voice cloning model may be a voice generation model pre-trained with a large-scale diversified voice dataset (e.g., a dataset including different genders, ages, accents, timbres, speech speeds, etc.). The model may have a wide range of generalization capabilities for different voice features, but is not specifically trained for the voice features of a particular user.

[0090] In some embodiments, the first target voice cloning model is a model that is further trained and fine-tuned based on the initial voice cloning model. The first target voice cloning model is a model that is customized and trained specifically for the voice characteristics of the target user through artificial intelligence (AI).

[0091] S23: when it is detected that the sound cloning instruction for the first target sound cloning model is triggered, initial voice data is acquired.

[0092] In some embodiments, when the voice cloning instruction for the first target voice cloning model is triggered, it indicates that the user (which may be the target user or other users) needs to convert specific voice data (such as initial voice data) into data of the timbre of the target user corresponding to the first target voice cloning model.

[0093] In some embodiments, the initial voice data may be a preset resource selected by the user, such as voice data such as voice stories and music stored locally on the terminal device or in a server, and the timbre of the initial voice data is the original timbre.

[0094] S24. Processing the initial voice data by using the first target voice cloning model to obtain target voice data corresponding to the target user, wherein the timbre of the target voice data is consistent with the timbre of the target user.

[0095] In some embodiments, the initial voice data may be input into the first target voice cloning model so that the timbre of the initial voice data is converted by the first target voice cloning model, and then the first target voice cloning model outputs the target voice data corresponding to the target user.

[0096] The embodiment of the present application obtains sample data corresponding to the target user, the sample data includes the voice information of the target user; trains a preset initial sound cloning model based on the sample data to obtain a first target sound cloning model corresponding to the target user; obtains initial voice data when it is detected that the sound cloning instruction for the first target sound cloning model is triggered; processes the initial voice data through the first target sound cloning model to obtain target voice data corresponding to the target user, and the timbre of the target voice data is consistent with the timbre of the target user. By collecting the user's voice to generate sample data and training the initial sound cloning model, an exclusive target sound cloning model corresponding to the user can be obtained; and then the target sound cloning model can be used to clone the specific initial voice data to obtain voice data consistent with the user's timbre, thereby enhancing the diversity of voice data and improving the level of personalization and customization of user experience.

[0097] In some embodiments, in S21, sample data corresponding to the target user is obtained, including S211-S213:

[0098] S211. When it is detected that the target user triggers a voice customization instruction, a first preset resource list is displayed, where the first preset resource list includes a plurality of first resource identifiers corresponding to a plurality of first preset resources.

[0099] In some embodiments, multiple first preset resources may include pre-stored voice data and / or corresponding text information, such as a voice story and a corresponding story content text. The text information can be used as a reference material for the target user to read aloud to collect the target user's voice information. The first preset resource and the first resource identifier correspond one-to-one, and the first resource identifier is used to uniquely indicate the corresponding first preset resource. The first resource identifier can be a resource name (such as a story name) and / or a number (such as a chapter number), etc.

[0100] Figure 3 A schematic diagram of a first page provided for an exemplary embodiment of the present application, such as Figure 3 As shown, the first page 31 is a page in the above application software, and the first preset resource list 311 can be displayed through the first page 31. In the first preset resource list 311, "000 Opening", "001 Liu Bei and Zhang Fei's First Acquaintance Part 1", "001 Liu Bei and Zhang Fei's First Acquaintance Part 2", and "002 Liu Bei, Guan Yu, and Zhang Fei's Oath of Brotherhood in Taoyuan Part 1" are multiple first resource identifiers corresponding to multiple first preset resources. In addition, a first expansion button with the words "Expand More Sounds" can also be displayed below the first preset resource list. When it is detected that the first expansion button is triggered, the first resource identifier that is not displayed in the current first preset resource list can be displayed.

[0101] Optionally, the first page 31 may also include the general titles of multiple first preset resources, such as "Romance of the Three Kingdoms", and related prompts, such as a prompt such as "Voice is not customized" to prompt the target user of the current customization status; "1 Record sound to collect your voice", "2 AI exclusive customization to restore your voice with AI", "3 Start playing and select a story below" to prompt the target user of the steps of voice cloning; "Choose a story and AI will tell it with your voice" is used to prompt the target user to select any one or more first resource identifiers in the above-mentioned first preset resource list as the first target resource identifier.

[0102] S212. When it is detected that the target user performs a confirmation operation on the first target resource identifier in the first preset resource list, the voice information sent by the target user to the target resource corresponding to the first target resource identifier is obtained, wherein the first target resource identifier is included in the multiple first resource identifiers.

[0103] In some embodiments, the target user's confirmation operation on the first target resource identifier in the first preset resource list may be that the target user clicks a display area where the first target resource identifier is located, and then clicks a preset customization button.

[0104] like Figure 3 The first page 31 may also include a customization button 312 displaying the words "Customize Now". When it is detected that the target user clicks on the display area where the first target resource identifier is located and clicks on the preset customization button 312, it can be determined to execute the above-mentioned step of obtaining the voice information sent by the above-mentioned target user for the target resource corresponding to the above-mentioned first target resource identifier.

[0105] In some embodiments, in the above S212, obtaining the voice information sent by the above target user for the target resource corresponding to the above first target resource identifier includes S2121-S2122:

[0106] S2121. Display a content page corresponding to the first target resource identifier, wherein the content page corresponding to the first target resource identifier includes a recording button and text content of the target resource corresponding to the first target resource identifier.

[0107] S2122: When a trigger operation on the recording button in the content page is detected, voice information sent by the target user to the target resource corresponding to the first target resource identifier is collected.

[0108] Figure 4 A schematic diagram of a content page provided for an exemplary embodiment of the present application, such as Figure 4, the content page 41 corresponding to the first target resource identifier may include a text information display area 411 for displaying the text content of the target resource corresponding to the first target resource identifier. The content page 41 may also include a recording button 412. When the triggering operation of the recording button 412 is detected, the microphone in the terminal device may be called to collect the voice information sent by the target user for the target resource corresponding to the first target resource identifier. The content page 41 may also include a trial listening button 413. When it is detected that the trial listening button 413 is triggered, the voice information recorded within the preset historical time length at the current moment is played. In addition, the content page 41 may also include a prompt such as "This text is about 200 words and the recording time is 1 minute."

[0109] In the embodiment of the present application, by displaying text content and a recording button on the content page, users can participate in the recording task more directly, such as recording according to a given text content. This direct interactive method increases user enthusiasm and simplifies the steps of user recording. It does not require users to make complex settings or jump to other interfaces, making it easy to use. By setting an audition button, users can immediately play the recorded voice information and check and modify their recording effects immediately. This real-time feedback function helps users to correct their recording content in a timely manner when necessary, thereby improving the accuracy and effect of the recording.

[0110] S213: Determine sample data according to the above voice information.

[0111] The sample data may include multiple voice messages of the target user. For example, when the target resource has a large amount of content, the voice message sent by the target user to the target resource may include multiple segments, such as the above Figure 4 The content page 41 may also include a switch button 414 (displaying the words "next paragraph"). When it is detected that the switch button 414 is triggered, the text content is turned to display other text contents after the current text content of the above-mentioned target resource, and then the voice information sent by the above-mentioned target user for the subsequent text content of the above-mentioned target resource is collected to obtain the voice information sent by the above-mentioned target user for the complete target resource.

[0112] In the embodiment of the present application, by presetting the resource list and clear prompts, the user is guided to gradually complete the sound customization operation, which reduces the user's learning cost. The user can easily select reference materials and customize the sound by clicking a simple button, and the operation process is smoother. In addition, by showing a list of multiple first preset resources (such as story content text), the user can freely select the material of interest to customize the sound, providing a personalized customization experience. This method can meet the personalized needs of different users and improve the user experience.

[0113] In some embodiments, in S213, sample data is determined according to the voice information, including S2131-S2132:

[0114] S2131. Determine a total amount of historical voice information corresponding to the target user based on the voice information, where the amount of historical voice information corresponding to the target user is the amount of voice information sent by the target user for the multiple first preset resources that has been currently acquired.

[0115] In some embodiments, the amount of historical voice information corresponding to the target user may be first obtained, and then the amount of historical voice information corresponding to the target user may be added with the currently collected voice information to obtain the total amount of historical voice information corresponding to the target user.

[0116] S2132: Determine whether the total number reaches a preset threshold. If so, pre-process the historical voice information and determine the pre-processed historical voice information as sample data.

[0117] The preprocessing may include any one or more of the following: noise reduction processing, smoothing processing, data cleaning, pre-emphasis processing, etc.

[0118] In the embodiment of the present application, by first calculating the total amount of historical voice information of the target user and determining whether it reaches a preset threshold, the amount of data to be processed can be effectively controlled. The preprocessing process helps to remove interference factors such as background noise, mutations, and distortion in the voice information, making the sample data clearer and more consistent. This high-quality data can improve the accuracy and robustness of subsequent model training and analysis.

[0119] In some embodiments, in the above S22, a preset initial sound cloning model is trained based on the above sample data to obtain a first target sound cloning model corresponding to the above target user, including S221-S223:

[0120] S221. Encode the sample data by using a sound encoder in a preset initial sound cloning model to obtain a first acoustic feature corresponding to the sample data.

[0121] In some embodiments, the initial voice cloning model may include a voice encoder and a speech synthesis model.

[0122] The sound encoder is used to extract and encode the timbre features in the input sample data (historical voice information), that is, the characteristic information of the speaker. The encoder can learn the features related to the speaker's unique timbre and pronunciation habits from the sample data corresponding to the speaker, without relying on the specific content. Optionally, the sound encoder can use a convolutional neural network, a long short-term memory network, or a network based on a self-attention mechanism to extract the features of the historical voice information.

[0123] In some embodiments, the first acoustic feature may be used to describe the physical and perceptual properties of the sample data corresponding to the target user, for example, the first acoustic feature may include a mel-spectrogram, a linear prediction coefficient, a fundamental frequency, a formant, etc. In the subsequent speech synthesis process, the first acoustic feature can help the speech synthesis model capture the timbre, intonation, pitch, rhythm and other information of the target user's speech.

[0124] S222. Fine-tune the speech synthesis model in the initial sound cloning model based on the first acoustic feature to obtain a fine-tuned speech synthesis model.

[0125] In some embodiments, a speech synthesis model may be used to convert input data (eg, speech data or text data) into natural and fluent speech output having specific characteristics.

[0126] The above-mentioned fine-tuning process is a technique in machine learning and deep learning, which refers to further training the model with a small amount of data based on a pre-trained model (initial sound cloning model) to make it perform better on specific tasks.

[0127] During the above-mentioned fine-tuning of the speech synthesis model, the relevant parameters of the speech synthesis model will be updated according to the first acoustic feature, so that the speech synthesis model can better adapt to the timbre and intonation of the target user. After the fine-tuning is completed, the speech synthesis model can generate a synthesized speech with a high degree of similarity to the target user's speech based on the input text and the acoustic features learned during the fine-tuning process. Through this fine-tuning method, the speech synthesis model can achieve personalized speech synthesis, making it more targeted and humane.

[0128] S223: Determine the trained initial voice cloning model as the first target voice cloning model corresponding to the target user.

[0129] The trained initial voice cloning model may include a voice encoder and the fine-tuned speech synthesis model.

[0130] In the embodiment of the present application, by fine-tuning the speech synthesis model in the initial sound cloning model, the speech synthesis model can learn the target user's unique voice features (such as timbre, pitch, speaking speed, etc.), so that the trained first target sound cloning model can generate more personalized and customized speech output.

[0131] In some embodiments, in S24, the initial voice data is processed by the first target voice cloning model to obtain the target voice data corresponding to the target user, including S241-S242:

[0132] S241. Input the initial voice data into the first target sound cloning model, and encode the initial voice data through the sound encoder in the first target sound cloning model to obtain a second acoustic feature corresponding to the initial voice data.

[0133] Among them, the second acoustic feature can represent the timbre, pronunciation, emotional expression and speaking style of the initial voice data.

[0134] S242. Process the initial voice data by using the fine-tuned voice synthesis model in the first target voice cloning model to obtain target voice data corresponding to the target user.

[0135] Through the fine-tuned speech synthesis model, the second acoustic features of the initial speech data will be adjusted and mapped to the target user's voice space, and the final output is the target speech data corresponding to the target user. These data retain the content of the input initial speech data (such as sentences or spoken content), but the sound features have been adjusted by the fine-tuned speech synthesis model to match the sound features of the target user.

[0136] In the embodiment of the present application, by encoding the initial voice data, extracting the specific acoustic features of the target user (such as timbre, pronunciation, emotional expression, speaking style, etc.), and applying these features to the speech synthesis model, it is possible to generate highly personalized synthesized speech, accurately imitate the voice characteristics of the target user, and make the generated speech more natural and realistic.

[0137] In some embodiments, in S22, training a preset initial sound cloning model based on the sample data to obtain a first target sound cloning model corresponding to the target user includes: sending the sample data to a server so that the server trains the preset initial sound cloning model based on the sample data to obtain the first target sound cloning model corresponding to the target user. That is, the first target sound cloning model can be stored in the server.

[0138] Further, in S24, the initial voice data is processed by the first target sound cloning model to obtain the target voice data corresponding to the target user, including: sending the initial voice data to the server, so that the server processes the initial voice data by the first target sound cloning model to obtain the target voice data corresponding to the target user; receiving the target voice data sent by the server.

[0139] By performing model training and voice data timbre conversion on the server side, local computing resources of the terminal device can be saved. The powerful computing power of the server can save time for model training and voice data conversion, thereby improving data processing efficiency.

[0140] In some embodiments, the data processing method further includes S201-S203:

[0141] S201: Display a second preset resource list, where the second preset resource list includes multiple second resource identifiers.

[0142] In some embodiments, the second resource identifier is used to indicate a corresponding preset resource, and the second resource identifier may be a resource name or number.

[0143] Figure 5 A schematic diagram of a second page provided for an exemplary embodiment of the present application, such as Figure 5 As shown, the second preset resource list 511 can be displayed in the second page 51, wherein the second page 51 can be a page in the above-mentioned application software.

[0144] In the second preset resource list 511, "000 Opening", "001 Liu Bei and Zhang Fei's First Acquaintance Part 1", "001 Liu Bei and Zhang Fei's First Acquaintance Part 2", and "002 Liu Bei, Guan Yu, and Zhang Fei's Peach Garden Oath Part 1" are multiple second resource identifiers. In addition, a second expansion button with the words "Expand More Sounds" can be displayed below the second preset resource list. When it is detected that the second expansion button is triggered, the second resource identifier that is not displayed in the current second preset resource list can be displayed.

[0145] Optionally, the second page 51 may also include the general titles of multiple second preset resources, such as "Romance of the Three Kingdoms", and related prompts, such as a prompt such as "The voice has been customized" to prompt the target user of the current customization status; "1 Record sound to collect your voice", "2 AI exclusive customization uses AI to restore your voice", "3 Start playing and select a story below" to prompt the target user of the steps of voice cloning; in addition, prompts such as "Select a story and AI will tell it with your voice" may also be displayed.

[0146] S202. When a confirmation operation for a second target resource identifier in the second preset resource list is detected, a plurality of model identifiers corresponding to the second target resource identifier are displayed, and a sound cloning model corresponding to each model identifier in the plurality of model identifiers is used to convert the input voice data into voice data consistent with the timbre of the corresponding user.

[0147] In some embodiments, each second resource identifier may correspond to one or more model identifiers, that is, for each second preset resource, there may be one or more corresponding sound cloning models, and multiple sound cloning models may be used to convert the second preset resource into speech data with different timbres.

[0148] S203. When a trigger operation is detected for a model identifier corresponding to the first target sound cloning model among the multiple model identifiers, it is determined that a sound cloning instruction for the first target sound cloning model is detected to be triggered, and the voice data corresponding to the second target resource identifier is used as the initial voice data.

[0149] The model identifier corresponding to the first target sound cloning model may be any one of the multiple model identifiers.

[0150] For example, a family scene may include a father, a mother, and a child. The "father" may correspond to the first target sound cloning model (model identifier A), and the mother may correspond to the second target sound cloning model (model identifier B). The multiple model identifiers corresponding to the second preset resource identifier "Romance of the Three Kingdoms" include A and B. When a trigger operation for the model identifier A corresponding to the first target sound cloning model is detected, it is determined that the sound cloning instruction for the first target sound cloning model corresponding to "father" is detected to be triggered, and the voice data corresponding to the second target resource identifier "Romance of the Three Kingdoms" (i.e., the voice content of the "Romance of the Three Kingdoms" story) is used as the above-mentioned initial voice data. The above-mentioned initial voice data can then be processed by the first target sound cloning model corresponding to "father" to obtain the target voice data corresponding to the above-mentioned target user "father", that is, the voice content of the "Romance of the Three Kingdoms" story told with the timbre of "father".

[0151] In the embodiment of the present application, by providing users with personalized different voice cloning models, users can choose to use specific voices to tell specific content, which effectively improves the personalization of resource playback and enhances user experience. And for each resource identifier (such as a story), multiple voice cloning models can be provided for users to choose, allowing users to choose different voices to present content according to different scenes or preferences, such as choosing the voice of "father" or "mother" to tell a story in a family scene, which enhances interactivity and fun.

[0152] Furthermore, the present application is described below in conjunction with specific embodiments. Figure 6 A specific flow chart of a data processing method provided by an exemplary embodiment of the present application is as follows: Figure 6 As shown, the data processing method includes the following S601-S611:

[0153] S601: When it is detected that the target user triggers a sound customization instruction, display a first preset resource list.

[0154] S602: When a confirmation operation of the target user on a first target resource identifier in the first preset resource list is detected, a content page corresponding to the first target resource identifier is displayed.

[0155] S603: When a trigger operation on a recording button in the content page is detected, voice information sent by the target user to the target resource corresponding to the first target resource identifier is collected.

[0156] S604: Determine the total amount of historical voice information corresponding to the target user based on the voice information.

[0157] S605: Determine whether the total number reaches a preset threshold. If so, execute S606; if not, return to execute the above S603.

[0158] S606: Preprocess the historical voice information, and determine the preprocessed historical voice information as sample data.

[0159] S607: Encode the sample data using a preset sound encoder in the initial sound cloning model to obtain a first acoustic feature corresponding to the sample data.

[0160] S608. Fine-tune the speech synthesis model in the initial sound cloning model based on the first acoustic feature to obtain a fine-tuned speech synthesis model.

[0161] S609: Determine the trained initial voice cloning model as the first target voice cloning model corresponding to the target user.

[0162] S610: Input the initial voice data into the first target sound cloning model, and encode the initial voice data through the sound encoder in the first target sound cloning model to obtain a second acoustic feature corresponding to the initial voice data.

[0163] S611. Process the initial voice data through the fine-tuned speech synthesis model in the first target voice cloning model to obtain target voice data corresponding to the target user.

[0164] The specific implementation of each step in the above data processing method has been described above and will not be repeated here.

[0165] Please refer to the following Figure 7 , which is a schematic diagram of the structure of a data processing device provided by an exemplary embodiment of the present application. Figure 7 As shown, the data processing device 700 includes:

[0166] A first acquisition module 701 is used to acquire sample data corresponding to a target user, wherein the sample data includes voice information of the target user;

[0167] A training module 702 is used to train a preset initial voice cloning model based on the sample data to obtain a first target voice cloning model corresponding to the target user;

[0168] A second acquisition module 703 is used to acquire initial voice data when it is detected that the voice cloning instruction for the first target voice cloning model is triggered;

[0169] The processing module 704 is used to process the initial voice data through the first target voice cloning model to obtain the target voice data corresponding to the target user, and the timbre of the target voice data is consistent with the timbre of the target user.

[0170] In a possible implementation, the first acquisition module 701 includes:

[0171] A display unit, configured to display a first preset resource list when it is detected that a target user triggers a sound customization instruction, wherein the first preset resource list includes a plurality of first resource identifiers corresponding to a plurality of first preset resources;

[0172] an acquisition unit, configured to acquire, when detecting a determination operation of the target user on a first target resource identifier in the first preset resource list, a voice message sent by the target user on a target resource corresponding to the first target resource identifier, wherein the first target resource identifier is included in the plurality of first resource identifiers;

[0173] The first determining unit is used to determine the sample data according to the above-mentioned voice information.

[0174] In a possible implementation, the acquisition unit includes:

[0175] A display subunit, used to display a content page corresponding to the first target resource identifier, wherein the content page corresponding to the first target resource identifier includes a recording button and text content of the target resource corresponding to the first target resource identifier;

[0176] The collection subunit is used to collect the voice information sent by the target user to the target resource corresponding to the first target resource identifier when a trigger operation on the recording button in the content page is detected.

[0177] In a possible implementation, the determining unit includes:

[0178] A determination subunit, configured to determine a total amount of historical voice information corresponding to the target user based on the voice information, wherein the amount of historical voice information corresponding to the target user is the amount of voice information sent by the target user for the plurality of first preset resources that has been currently acquired;

[0179] The judging subunit is used to judge whether the above total number reaches a preset threshold value. If so, the above historical voice information is pre-processed and the pre-processed historical voice information is determined as sample data.

[0180] In a possible implementation, the training module 702 includes:

[0181] A first encoding unit, configured to encode the sample data by using a sound encoder in a preset initial sound cloning model to obtain a first acoustic feature corresponding to the sample data;

[0182] A fine-tuning unit, configured to fine-tune the speech synthesis model in the initial sound cloning model based on the first acoustic feature to obtain a fine-tuned speech synthesis model;

[0183] The second determining unit is used to determine the trained initial voice cloning model as the first target voice cloning model corresponding to the target user.

[0184] In a possible implementation, the processing module 704 includes:

[0185] a second encoding unit, configured to input the initial voice data into the first target voice cloning model, and encode the initial voice data through a voice encoder in the first target voice cloning model to obtain a second acoustic feature corresponding to the initial voice data;

[0186] The processing unit is used to process the initial voice data through the fine-tuned speech synthesis model in the first target voice cloning model to obtain the target voice data corresponding to the target user.

[0187] In a possible implementation, the above device further includes:

[0188] A display module, used for displaying a second preset resource list, wherein the second preset resource list includes a plurality of second resource identifiers;

[0189] The display module is further configured to display a plurality of model identifiers corresponding to the second target resource identifier in the second preset resource list when a determination operation is detected for the second target resource identifier, wherein the sound cloning model corresponding to each model identifier in the plurality of model identifiers is configured to convert the input voice data into voice data consistent with the timbre of the corresponding user;

[0190] A determination module is used to determine that a sound cloning instruction for the first target sound cloning model is triggered when a trigger operation is detected for a model identifier corresponding to the first target sound cloning model among the multiple model identifiers, and to use the voice data corresponding to the second target resource identifier as the initial voice data.

[0191] The division of the modules in the above-mentioned data processing device 700 is only for illustration. In other embodiments, the data processing device can be divided into different modules as needed to complete all or part of the functions of the above-mentioned data processing device. The implementation of each module in the data processing device provided in the embodiments of this specification can be in the form of a computer program. The computer program can be run on a terminal or a server. The program modules constituted by the computer program can be stored in the memory of the terminal or the server. When the computer program is executed by the processor, all or part of the steps of the data processing method described in the embodiments of this specification are implemented.

[0192] See next Figure 8 , which is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. Figure 8 As shown, the electronic device 800 may include: a processor 810 and a memory 820 , and may also include a user interface 830 , a network interface 840 and a communication bus 850 .

[0193] Among them, the processor 810 may include one or more processing cores. The processor 810 uses various interfaces and lines to connect various parts of the entire electronic device 800, and executes various functions and processes data of the electronic device 800 by running or executing instructions, programs, code sets or instruction sets stored in the memory 820, and calling data stored in the memory 820. Optionally, the processor 810 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 810 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 810, and it can be implemented separately through a chip.

[0194] The memory 820 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 820 includes a non-transitory computer-readable storage medium. The memory 820 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 820 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a receiving function, a control function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 820 may optionally be at least one storage device located away from the aforementioned processor 810. As Figure 8 As shown, the memory 820 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

[0195] Optionally, the communication bus 850 is used to realize the connection and communication between these components. The user interface 830 may include a display screen (Display), a camera (Camera), and may also include a standard wired interface and a wireless interface; the network interface 840 may optionally include a standard wired interface, a wireless interface (such as interface).

[0196] exist Figure 8 In the electronic device 800 shown, the processor 810 may be used to call the program instructions stored in the memory 820 and specifically perform the following operations:

[0197] Acquire sample data corresponding to the target user, the sample data including voice information of the target user;

[0198] Training a preset initial voice cloning model based on the sample data to obtain a first target voice cloning model corresponding to the target user;

[0199] When it is detected that the voice cloning instruction for the first target voice cloning model is triggered, initial voice data is acquired;

[0200] The initial voice data is processed by the first target voice cloning model to obtain target voice data corresponding to the target user, and the timbre of the target voice data is consistent with the timbre of the target user.

[0201] In a possible implementation, the step of obtaining sample data corresponding to the target user includes:

[0202] When it is detected that the target user triggers the sound customization instruction, a first preset resource list is displayed, where the first preset resource list includes a plurality of first resource identifiers corresponding to a plurality of first preset resources;

[0203] When detecting a determination operation of the target user on a first target resource identifier in the first preset resource list, obtaining a voice message sent by the target user on a target resource corresponding to the first target resource identifier, wherein the first target resource identifier is included in the plurality of first resource identifiers;

[0204] The sample data is determined according to the above voice information.

[0205] In a possible implementation, the obtaining of the voice information sent by the target user to the target resource corresponding to the first target resource identifier includes:

[0206] Displaying a content page corresponding to the first target resource identifier, wherein the content page corresponding to the first target resource identifier includes a recording button and text content of the target resource corresponding to the first target resource identifier;

[0207] When a trigger operation on the recording button in the content page is detected, voice information sent by the target user to the target resource corresponding to the first target resource identifier is collected.

[0208] In a possible implementation, the determining of sample data according to the voice information includes:

[0209] Determine the total amount of historical voice information corresponding to the target user based on the voice information, where the amount of historical voice information corresponding to the target user is the amount of voice information sent by the target user for the multiple first preset resources that has been currently acquired;

[0210] It is determined whether the total number reaches a preset threshold. If so, the historical voice information is preprocessed and the preprocessed historical voice information is determined as sample data.

[0211] In a possible implementation, the training of a preset initial voice cloning model based on the sample data to obtain a first target voice cloning model corresponding to the target user includes:

[0212] Encoding the sample data by a sound encoder in a preset initial sound cloning model to obtain a first acoustic feature corresponding to the sample data;

[0213] Fine-tuning the speech synthesis model in the initial sound cloning model based on the first acoustic feature to obtain a fine-tuned speech synthesis model;

[0214] The trained initial voice cloning model is determined as the first target voice cloning model corresponding to the target user.

[0215] In a possible implementation, the processing of the initial voice data by the first target voice cloning model to obtain the target voice data corresponding to the target user includes:

[0216] Inputting the initial voice data into the first target sound cloning model, encoding the initial voice data through a sound encoder in the first target sound cloning model, and obtaining a second acoustic feature corresponding to the initial voice data;

[0217] The initial voice data is processed by the fine-tuned voice synthesis model in the first target voice cloning model to obtain the target voice data corresponding to the target user.

[0218] In a possible implementation, the method further includes:

[0219] Displaying a second preset resource list, wherein the second preset resource list includes a plurality of second resource identifiers;

[0220] When a determination operation is detected for a second target resource identifier in the second preset resource list, a plurality of model identifiers corresponding to the second target resource identifier are displayed, wherein the sound cloning model corresponding to each model identifier in the plurality of model identifiers is used to convert the input voice data into voice data consistent with the timbre of the corresponding user;

[0221] When a trigger operation is detected for a model identifier corresponding to the first target sound cloning model among the multiple model identifiers, it is determined that the sound cloning instruction for the first target sound cloning model is triggered, and the voice data corresponding to the second target resource identifier is used as the initial voice data.

[0222] The embodiment of the present application also provides a computer-readable storage medium, which stores instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes one or more steps in the above embodiment. If the components of the above data processing device are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium.

[0223] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The above-mentioned computer program product includes one or more computer instructions. When the above-mentioned computer program instructions are loaded and executed on a computer, the above-mentioned process or function according to the embodiment of the present application is generated in whole or in part. The above-mentioned computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium or transmitted by the above-mentioned computer-readable storage medium. The above-mentioned computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The above-mentioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The above-mentioned available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, digital versatile discs (DVD)), or semiconductor media (for example, solid state disks (SSD)), etc.

[0224] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes. In the absence of conflict, the technical features in this embodiment and the implementation scheme can be combined arbitrarily.

[0225] The above-mentioned embodiments are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the protection scope determined by the claims of the present application.

Claims

1. A data processing method, characterized in that: include: Acquire sample data corresponding to a target user, wherein the sample data includes voice information of the target user; Training a preset initial voice cloning model based on the sample data to obtain a first target voice cloning model corresponding to the target user; When it is detected that the sound cloning instruction for the first target sound cloning model is triggered, acquiring initial voice data; The initial voice data is processed by the first target voice cloning model to obtain target voice data corresponding to the target user, and the timbre of the target voice data is consistent with the timbre of the target user.

2. The method according to claim 1, characterized in that The obtaining of sample data corresponding to the target user includes: When it is detected that the target user triggers the sound customization instruction, a first preset resource list is displayed, where the first preset resource list includes a plurality of first resource identifiers corresponding to a plurality of first preset resources; When a determination operation of the target user on a first target resource identifier in the first preset resource list is detected, obtaining voice information sent by the target user on a target resource corresponding to the first target resource identifier, wherein the first target resource identifier is included in the multiple first resource identifiers; Sample data is determined according to the voice information.

3. The method according to claim 2, characterized in that The acquiring of the voice information sent by the target user to the target resource corresponding to the first target resource identifier includes: Displaying a content page corresponding to the first target resource identifier, wherein the content page corresponding to the first target resource identifier includes a recording button and text content of the target resource corresponding to the first target resource identifier; When a trigger operation on a recording button in the content page is detected, voice information uttered by the target user with respect to the target resource corresponding to the first target resource identifier is collected.

4. The method according to claim 3, characterized in that The determining of sample data according to the voice information comprises: Determine the total amount of historical voice information corresponding to the target user based on the voice information, where the amount of historical voice information corresponding to the target user is the amount of voice information sent by the target user for the multiple first preset resources that has been currently acquired; It is determined whether the total number reaches a preset threshold value. If so, the historical voice information is preprocessed and the preprocessed historical voice information is determined as sample data.

5. The method according to claim 1, characterized in that The step of training a preset initial voice cloning model based on the sample data to obtain a first target voice cloning model corresponding to the target user includes: Encoding the sample data by using a sound encoder in a preset initial sound cloning model to obtain a first acoustic feature corresponding to the sample data; Fine-tuning the speech synthesis model in the initial sound cloning model based on the first acoustic feature to obtain a fine-tuned speech synthesis model; The trained initial voice cloning model is determined as a first target voice cloning model corresponding to the target user.

6. The method according to claim 5, characterized in that The processing of the initial voice data by using the first target voice cloning model to obtain target voice data corresponding to the target user includes: Inputting the initial voice data into the first target sound cloning model, encoding the initial voice data through a sound encoder in the first target sound cloning model, and obtaining a second acoustic feature corresponding to the initial voice data; The initial voice data is processed by the fine-tuned speech synthesis model in the first target sound cloning model to obtain target voice data corresponding to the target user.

7. The method according to claim 1, characterized in that The method further comprises: Displaying a second preset resource list, wherein the second preset resource list includes a plurality of second resource identifiers; When a determination operation is detected for a second target resource identifier in the second preset resource list, a plurality of model identifiers corresponding to the second target resource identifier are displayed, wherein the sound cloning model corresponding to each model identifier in the plurality of model identifiers is used to convert the input voice data into voice data consistent with the timbre of the corresponding user; When a trigger operation is detected for a model identifier corresponding to the first target sound cloning model among the multiple model identifiers, it is determined that a sound cloning instruction for the first target sound cloning model is detected to be triggered, and the voice data corresponding to the second target resource identifier is used as the initial voice data.

8. A data processing device, characterized in that: include: A first acquisition module, used to acquire sample data corresponding to a target user, wherein the sample data includes voice information of the target user; A training module, configured to train a preset initial voice cloning model based on the sample data to obtain a first target voice cloning model corresponding to the target user; A second acquisition module is configured to acquire initial voice data when it is detected that a voice cloning instruction for the first target voice cloning model is triggered; The processing module is used to process the initial voice data through the first target voice cloning model to obtain target voice data corresponding to the target user, and the timbre of the target voice data is consistent with the timbre of the target user.

9. An electronic device, characterized in that: include: Processor and memory; The memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the steps of the method according to any one of claims 1 to 7.

Images