A model training method, a voice playback method, a device and a storage medium

Abstract

The present application discloses a model training method, a voice playback method, an apparatus and a storage medium, which are applied to the field of artificial intelligence cloud services. The method of the present application includes: obtaining a set of voice data to be trained; when N meets the threshold of the number of voice training, obtaining a voice model training instruction; in response to the voice model training instruction, based on the voice data set to be trained, obtaining a predicted voice data set through the voice model to be trained; based on the predicted voice data set and the voice data set to be trained, training the voice model to be trained to obtain a target voice model; sending the target voice model so that the terminal device stores the target voice model. The present application improves the flexibility of voice model generation and meets the user's customization needs for sound. Secondly, the possibility of voice model selection is improved when the user plays voice, thereby improving the flexibility of voice playback, and then improving the user's voice playback experience and flexibility.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence cloud services, and in particular to a model training method, a voice playback method, a device and a storage medium. Background Art

[0002] With the rapid development of Internet information technology and the continuous improvement of the quality of life, intelligent terminal devices are widely used in people's lives, and users' demand for intelligent terminal devices is also increasing. In order to meet the personalized experience of users on different terminal devices, a method of completing voice playback based on artificial intelligence has emerged.

[0003] At present, the method of voice playback is usually to integrate the voice model data of the speaker (such as a public star) into the terminal device. The user selects the desired speaker, and the terminal device synthesizes the voice of the speaker selected by the user with the text information of the required voice playback to obtain the target voice data, so that the terminal device plays the target voice data, thereby completing the voice playback.

[0004] However, since the voice model is integrated into the terminal device, the user can only select the voice model integrated into the terminal device, which reduces the selectivity and flexibility of the voice model, thereby reducing the flexibility of voice playback. Summary of the invention

[0005] The embodiments of the present application provide a model training method, a voice playback method, an apparatus and a storage medium for improving the flexibility of voice model generation, and improving the possibility of voice model selection when the user plays the voice, thereby improving the flexibility of voice playback.

[0006] In view of this, the present application provides a model training method, comprising:

[0007] Acquire a speech data set to be trained, wherein the speech data set to be trained includes N speech data to be trained, where N is a positive integer greater than or equal to 1;

[0008] When N meets the speech training quantity threshold, obtain the speech model training instruction;

[0009] In response to the speech model training instruction, based on the speech data set to be trained, a predicted speech data set is obtained through the speech model to be trained, wherein the predicted speech data set includes N predicted speech data, and the predicted speech data has a corresponding relationship with the speech data to be trained;

[0010] Based on the predicted speech data set and the speech data set to be trained, the speech model to be trained is trained to obtain a target speech model;

[0011] The target speech model is sent so that the terminal device stores the target speech model.

[0012] Another aspect of the present application provides a method for voice playback, comprising:

[0013] Detecting an operation for voice playback, sending a voice playback request, so that the server obtains service content text data based on the voice data to be parsed, and obtains timbre-free voice data according to the service content text data, wherein the voice playback request carries the voice data to be parsed, the service content text data is obtained according to the voice data to be parsed, and the timbre-free voice data is obtained after voice synthesis of the service content text data;

[0014] Receiving timbre-free voice data;

[0015] Generate target speech data according to the timbre-free speech data and the target speech model, wherein the target speech model is obtained by training the speech model to be trained based on the predicted speech data set and the speech data set to be trained, and the predicted speech data set is obtained by the server in response to the speech model training instruction, based on the speech data set to be trained, through the speech model to be trained;

[0016] Play the target voice data.

[0017] On the other hand, the present application provides a model training device, comprising:

[0018] An acquisition module is used to acquire a speech data set to be trained, wherein the speech data set to be trained includes N speech data to be trained, where N is a positive integer greater than or equal to 1;

[0019] The acquisition module is also used to obtain the speech model training instruction when N meets the speech training quantity threshold;

[0020] The acquisition module is further used to respond to the speech model training instruction and, based on the speech data set to be trained, acquire a predicted speech data set through the speech model to be trained, wherein the predicted speech data set includes N predicted speech data, and the predicted speech data has a corresponding relationship with the speech data to be trained;

[0021] A training module, used for training the speech model to be trained based on the predicted speech data set and the speech data set to be trained to obtain a target speech model;

[0022] The sending module is used to send the target speech model so that the terminal device stores the target speech model.

[0023] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0024] An acquisition module, specifically used to acquire a voice service request;

[0025] In response to the voice service request, a voice data collection instruction is sent to the terminal device, so that the terminal device responds to the voice data collection instruction and obtains the voice data to be detected, wherein the voice data to be detected is obtained based on the voice training text, and the voice training text is obtained according to the voice data collection instruction;

[0026] When the speech data to be detected meets the speech training qualification condition, the speech data to be trained is determined;

[0027] Based on the speech data to be trained, a speech data set to be trained is generated.

[0028] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0029] The training module is specifically used to update the model parameters of the speech model to be trained according to the target loss function based on the predicted speech data set and the speech data set to be trained;

[0030] If the target loss function reaches convergence, the target speech model is generated according to the model parameters.

[0031] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0032] The acquisition module is also used to obtain a voice playback request, wherein the voice playback request carries the voice data to be parsed;

[0033] The acquisition module is further used to acquire service content text data according to the voice data to be parsed, wherein the service content text data corresponds to the voice data to be parsed;

[0034] The acquisition module is further used to acquire the timbre-free voice data according to the service content text data, wherein the timbre-free voice data is obtained by performing voice synthesis on the service content text data;

[0035] The sending module is also used to send the timbre-free voice data to the terminal device, so that the terminal device generates target voice data according to the timbre-free voice data and the target voice model, and plays the target voice data.

[0036] Another aspect of the present application provides a voice playback device, comprising:

[0037] A sending module, used for detecting an operation for voice playback, sending a voice playback request, so that the server obtains service content text data based on the voice data to be parsed, and obtains timbre-free voice data according to the service content text data, wherein the voice playback request carries the voice data to be parsed, the service content text data is obtained according to the voice data to be parsed, and the timbre-free voice data is obtained by performing voice synthesis on the service content text data;

[0038] An acquisition module, used for acquiring voice data without timbre;

[0039] A generating module, for generating target speech data according to the timbre-free speech data and the target speech model, wherein the target speech model is obtained by training the speech model to be trained based on the predicted speech data set and the speech data set to be trained, and the predicted speech data set is obtained by the server in response to the speech model training instruction, based on the speech data set to be trained, through the speech model to be trained;

[0040] The playing module is used to play the target voice data.

[0041] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0042] The sending module is also used to detect operations for voice services and send voice service requests;

[0043] The acquisition module is further used to acquire the voice data collection instruction when the server responds to the voice service request;

[0044] The acquisition module is further used to respond to the voice data collection instruction, and when the voice data to be detected meets the noise qualification condition, obtain the voice data to be detected, wherein the voice data to be detected is obtained based on the voice training text, and the voice training text is obtained according to the voice data collection instruction;

[0045] The sending module is also used to send the voice data to be detected, so that the server generates a voice data set to be trained based on the voice data to be detected, and the voice data set to be trained includes N voice data to be trained, where N is a positive integer greater than or equal to 1.

[0046] In a possible design, in another implementation of another aspect of the embodiment of the present application, the voice playback device further includes a storage module.

[0047] The sending module is further used to send a speech model training instruction when N meets the speech training quantity threshold, so that the server obtains the target speech model;

[0048] The acquisition module is also used to acquire the target speech model;

[0049] The storage module is used to store the target speech model.

[0050] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0051] The acquisition module is further used to acquire voice data collection status information, wherein the voice data collection status information is used to indicate that the voice data to be detected is being acquired;

[0052] The acquisition module is further used to acquire the voice data collection completion status information, wherein the voice data collection completion status information is used to indicate that the voice data has been acquired and the target voice model can be trained;

[0053] The acquisition module is further used to acquire the target speech model training status information, wherein the target speech model training status information is used to indicate that the speech model to be trained is being trained;

[0054] The acquisition module is also used to obtain the target speech model completion status information, wherein the target speech model completion status information is used to indicate that the target speech model has completed training.

[0055] Another aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the computer-readable storage medium is run on a computer, the computer executes the methods described in the above aspects.

[0056] Another aspect of the present application provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the methods provided in various optional implementations of the above-mentioned various aspects.

[0057] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0058] In an embodiment of the present application, a method for model training is provided, wherein the server first obtains a speech data set to be trained including N speech data to be trained, and when N meets the speech training quantity threshold, a speech model training instruction is obtained, and then in response to the speech model training instruction, based on the speech data set to be trained, a predicted speech data set is obtained through the speech model to be trained, and further, based on the predicted speech data set and the speech data set to be trained, the speech model to be trained is trained to obtain a target speech model, and the target speech model is sent to a terminal device so that the terminal device stores the target speech model. In the above manner, since the target speech model is obtained by training the speech model to be trained based on the predicted speech data set and the speech data set to be trained, the terminal device can obtain the target speech model required by the user from the speech data to be trained provided by the user, thereby improving the flexibility of speech model generation and meeting the user's customization requirements for sound. Secondly, the possibility of speech model selection is improved when the user plays speech, thereby improving the flexibility of speech playback, and further improving the user's speech playback experience and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of an architecture for model training in an embodiment of the present application;

[0060] Figure 2 Another schematic diagram of the architecture of the model training in the embodiment of the present application;

[0061] Figure 3 A flowchart of the method for model training in an embodiment of the present application;

[0062] Figure 4 This is a schematic diagram of an embodiment of the method for model training in the embodiments of the present application;

[0063] Figure 5 This is another flow chart of the method for model training in the embodiment of the present application;

[0064] Figure 6 A flowchart of a method for voice playback in an embodiment of the present application;

[0065] Figure 7 A schematic diagram of an embodiment of the method for voice playback in an embodiment of the present application;

[0066] Figure 8 This is a schematic diagram of an embodiment of the operation for voice playback in an embodiment of the present application;

[0067] Fig. 9 This is a schematic diagram of another embodiment of the method for voice playback in the embodiment of the present application;

[0068] Fig.10This is another flowchart of the method for voice playback in an embodiment of the present application;

[0069] Fig.11 This is a schematic diagram of another embodiment of the method for voice playback in the embodiment of the present application;

[0070] Fig.12 This is a schematic diagram of an embodiment of a model training device in an embodiment of the present application;

[0071] Fig.13 This is a schematic diagram of an embodiment of a voice playback device in an embodiment of the present application;

[0072] Fig.14 This is a schematic diagram of an embodiment of a server in an embodiment of the present application;

[0073] Fig.15 This is a schematic diagram of an embodiment of a terminal device in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The embodiments of the present application provide a model training method, a voice playback method, an apparatus and a storage medium for improving the flexibility of voice model generation and meeting the user's customization needs for sound. Secondly, the possibility of voice model selection is improved when the user plays voice, thereby improving the flexibility of voice playback, and further improving the user's voice playback experience and flexibility.

[0075] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0076] With the rapid development of Internet information technology and the continuous improvement of the quality of life, intelligent terminal devices are widely used in people's lives, and users' demand for intelligent terminal devices is also increasing. In order to meet the personalized experience of users on different terminal devices, a method of completing voice playback based on artificial intelligence has been created. The method provided in this application can be applied to various scenarios in the field of artificial intelligence cloud services, for example, when the driver requests voice navigation from the terminal device while driving, or when the infant terminal device reads a bedtime story by voice, or when the hands perform other daily operations, the terminal device voice plays news or novels, etc. It should be understood that in actual applications, artificial intelligence cloud services can also be applied to many other scenarios, so the application scenarios are not exhaustively listed here.

[0077] In order to improve the flexibility of voice playback in the above-mentioned scenario, this application proposes a model training method and a voice playback method, which are applied to Figure 1 The model training system shown in Figure 1 , Figure 1 This is an architectural diagram of the model training system in the embodiment of the present application. As shown in the figure, the model training system includes a server and a terminal device, and the client is deployed on the terminal device. The server involved in the present application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication. Secondly, although Figure 1 Only six terminal devices, a network and a server are shown, but it should be understood that Figure 1 The examples are only used to understand this solution. The number of specific terminal devices, networks and servers should be flexibly determined based on actual conditions.

[0078] Specifically, before the user plays the target voice data, it is necessary to first obtain the target voice model, the terminal device detects the operation for the voice service, and sends a voice service request to the server. The server responds to the voice service request and sends a voice data collection instruction to the terminal device. The terminal device responds to the voice data collection instruction to obtain the voice data to be detected, and when the voice data to be detected meets the noise qualification condition, the voice data to be detected is sent to the server. The server generates a set of voice data to be trained including N voice data to be trained based on the voice data to be trained. When N meets the voice training quantity threshold, the voice model training instruction sent by the terminal device is obtained, and the server responds to the voice model training instruction. A type of training instruction is provided, based on the speech data set to be trained, a predicted speech data set is obtained through the speech model to be trained, and then based on the predicted speech data set and the speech data set to be trained, the speech model to be trained is trained to obtain a target speech model, and finally the server sends the target speech model to the terminal device so that the terminal device stores the target speech model, and when the target speech data needs to be played, the operation for speech playback is detected, and a speech playback request is sent to the server so that the server obtains the timbre-free speech data and sends it to the terminal device, and the terminal device can generate the target speech data according to the timbre-free speech data and the target speech model, and play the target speech data. Since the target speech model is obtained by training the speech model to be trained based on the predicted speech data set and the speech data set to be trained, the terminal device can obtain the corresponding target speech model from the speech data to be trained provided by the user, thereby improving the flexibility of speech model generation, and the possibility of speech model selection is improved when the user plays speech, thereby improving the flexibility of speech playback.

[0079] For further information, see Figure 2 , Figure 2 This is another architecture diagram of the model training system in the embodiment of the present application. As shown in the figure, the terminal device is integrated with Tencent Voice Service (TVS)-Software Development Kit (SDK) and speech synthesis technology (TTS)-SDK, and the TVS-SDK is integrated with a model management module and a message module, while the TTS-SDK is integrated with a model management module and sound synthesis. Specifically, TVS-SDK and TTS-SDK can interact with the TVS service in the server, and the model management module integrated in the TVS-SDK interacts with the TVS-web service in the server, which is specifically used for the transmission of voice data, model data and other data. The message module integrated in the TVS-SDK interacts with the message push service in the server to realize the push of various messages.

[0080] Secondly, the server includes TVS service, TVS-web service, TTS customization service, offline training service, message push service and database. TVS service can provide voice services such as automatic speech recognition technology (automatics speech The terminal device obtains TVS-specific services through semantic query, such as querying news, novels, playing music, navigation and other services. The TVS-web service is a gateway service that provides a non-voice interaction interface to the outside. The voice customization interface is provided through this service. The TTS customization service is responsible for receiving the training request initiated by the terminal device, collecting the voice data of the terminal device, creating and triggering the offline training task, and writing the task record into the database, returning the task submission success status, and constantly querying the task status. If the task is completed, the message push service sends a message to the terminal device to notify the terminal device that the target model training has been completed. The offline training service uses the AI ​​deep learning method to extract features and train the training voice data, and finally obtains the target voice model, and updates the database to save the target training model data, and the task status is changed to a successful state. The message push service has a channel for pushing messages to the terminal device. The channel capability is used in the embodiment of the present application to enable the terminal device to know the training result of the target voice model. Therefore, after the terminal device receives the message pushed by the message push server in the server, it jumps to the model page through the message, selects the target voice model, and can trigger TTS-SDK to store the trained target voice model locally. When the terminal device triggers the TVS service, such as novel broadcasting, TTS-SDK combines the target voice model and the toneless voice data synthesized by the TTS data in the server to generate the target voice data, and plays the target voice data to realize voice broadcasting.

[0081] Since the embodiment of the present application is applied to the field of cloud technology, before the method of voice playback provided by the embodiment of the present application is specifically introduced, some basic concepts of cloud technology are introduced first. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize the calculation, storage, processing, and sharing of data. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the application of cloud computing business model, which can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background service of the technical network system requires a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites. With the high development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data need strong system backing support, which can only be achieved through cloud computing.

[0082] With the research and progress of cloud technology, cloud technology has been studied in many directions, and artificial intelligence (AI) cloud services are generally referred to as artificial intelligence as a service (AIaaS). This is the current mainstream service mode of artificial intelligence platforms. Specifically, the AIaaS platform will split several common AI services and provide independent or packaged services in the cloud. This service model is similar to opening an AI theme mall. All developers can access and use one or more artificial intelligence services provided by the platform through the application programming interface (API). Some senior developers can also use the AI ​​framework and AI infrastructure provided by the platform to deploy and operate their own cloud artificial intelligence services.

[0083] Secondly, AI cloud services require AI application fields, so some basic concepts of AI are introduced. AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. AI is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. AI technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. AI basic technologies generally include technologies such as sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. AI software technology mainly includes several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. Machine Learning (ML) is a multi-disciplinary cross-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It is a field that studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0084] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied in many directions. The key technologies of speech technology include automatic speech recognition technology (ASR), speech synthesis technology (TTS) and voiceprint recognition technology. Enabling computers to listen, see, speak and feel is the development direction of human-computer interaction in the future, among which speech has become one of the most promising human-computer interaction methods in the future. Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. Therefore, research in this field will involve natural language, that is, the language used by people in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0085] Based on this, the following will introduce the method of voice playback. Figure 3 , Figure 3 The following is a flow chart of the method for model training in the embodiment of the present application, as shown in the figure, specifically:

[0086] In step S1, the terminal device sends voice data to be trained to a server, where the voice data to be trained is voice data acquired by a user through voice input.

[0087] In step S2, when the amount of speech data to be trained sent by the terminal device to the server meets the speech training amount threshold, the terminal device may also send a speech model training instruction to the server.

[0088] In step S3, the server can obtain a set of speech data to be trained including N speech data to be trained through the aforementioned steps, and then the server responds to the speech model training instruction, uses the set of speech data to be trained as the input of the speech model to be trained, and obtains a set of predicted speech data including N predicted speech data through the speech model to be trained.

[0089] In step S4, the server trains the speech model to be trained based on the predicted speech data set and the speech data set to be trained to obtain a target speech model.

[0090] In step S5, the server sends the target speech model to the terminal device.

[0091] In step S6, the terminal device stores the target speech model.

[0092] In step S7, when the user needs to play voice, the terminal device detects the operation for voice playing, and generates a voice playing request carrying the voice data to be parsed through voice input, and sends the voice playing request to the server.

[0093] In step S8, the server obtains service content text data based on the voice data to be parsed carried in the voice play request, and obtains timbre-free voice data according to the service content text data.

[0094] In step S9, the server sends the toneless voice data to the terminal device.

[0095] In step S10, after acquiring the timbre-free speech data, the terminal device generates target speech data according to the acquired timbre-free speech data and the stored target speech model.

[0096] In step S11, target voice data is played.

[0097] The solution provided in the embodiment of the present application relates to artificial intelligence cloud service technology in cloud technology. In combination with the above introduction, the method of voice playback in the present application will be introduced below. Please refer to Figure 4 , Figure 4 This is a schematic diagram of an embodiment of the method for model training in an embodiment of the present application. As shown in the figure, an embodiment of the method for model training in an embodiment of the present application includes:

[0098] 101. Obtain a speech data set to be trained, wherein the speech data set to be trained includes N speech data to be trained, where N is a positive integer greater than or equal to 1;

[0099] In this embodiment, the server obtains a set of speech data to be trained including N speech data to be trained, that is, it can receive the set of speech data to be trained sent by the terminal device in real time, or obtain the set of speech data to be trained from the database of the terminal device, or obtain the set of speech data to be trained from the database of the server. The specific method of obtaining the set of speech data to be trained is not limited here.

[0100] It should be understood that after the server obtains the speech data set to be trained through step 101, if the user needs to train other speech models, the server can continue to obtain the speech data set required for other speech models, that is, there is no need to complete the training of the target speech model before starting to obtain it.

[0101] 102. When N meets the speech training quantity threshold, obtain a speech model training instruction;

[0102] In this embodiment, when the number of voice data to be trained in the voice data set to be trained reaches the voice training number threshold, the terminal device can obtain the voice data collection completion status information. According to the voice data collection completion status information, it can be known that the voice data of the terminal device has been acquired and the target voice model can be trained. Therefore, a voice model training instruction is sent to the server, and the server can obtain the voice model training instruction sent by the terminal device.

[0103] Specifically, in this embodiment, the threshold value of the number of speech training is 20, that is, after the speech data set to be trained includes 20 speech data to be trained, the terminal device can send a speech model training instruction to the server, and the server obtains the speech model training instruction. It should be understood that in actual applications, the threshold value of the number of speech training can also be other positive integers such as 15, 25 and 30. The value of the specific threshold value of the number of speech training should be flexibly determined according to the actual situation and needs of the speech model to be trained.

[0104] 103. In response to the speech model training instruction, based on the speech data set to be trained, a predicted speech data set is obtained through the speech model to be trained, wherein the predicted speech data set includes N predicted speech data, and the predicted speech data has a corresponding relationship with the speech data to be trained;

[0105] In this embodiment, after the server receives the speech model training instruction obtained in step 102, it can be known that the terminal device needs to train the speech model to be trained. Therefore, it can respond to the speech model training instruction and use the speech data set to be trained obtained in step 101 as the input of the speech model to be trained. The speech model to be trained can output a predicted speech data set including N predicted speech data, and the predicted speech data has a corresponding relationship with the speech data to be trained.

[0106] Exemplarily, the speech data set to be trained includes speech data A to speech data T to be trained, a total of 20 speech data to be trained. Then, after the speech data set to be trained is used as the input of the speech model to be trained, the predicted speech data set output by the speech model to be trained includes predicted speech data A to predicted speech data T, a total of 20 predicted speech data, and the speech data A to be trained corresponds to the predicted speech data A, that is, the speech data A to be trained is used as the input of the speech model to be trained alone, the speech model to be trained can output the predicted speech data A, and the speech data T to be trained also corresponds to the predicted speech data T, and other corresponding relationships are similar to the above, and will not be repeated in detail.

[0107] 104. Based on the predicted speech data set and the speech data set to be trained, the speech model to be trained is trained to obtain a target speech model;

[0108] In this embodiment, the server uses an offline training module to perform feature extraction using a deep learning algorithm in combination with the predicted speech data set obtained in step 103 and the speech data set to be trained obtained in step 101, and trains the speech model to be trained. When the training conditions are met, the target speech model can be obtained, and after the training is completed, the obtained target speech model is stored in a database in the server.

[0109] For example, if the user uses the voice of "Dad" as the voice data to be trained, then the target voice model obtained through training can be the voice model corresponding to the voice of "Dad". If the user uses the voice of "Mom" as the voice data to be trained, then the target voice model obtained through training can be the voice model corresponding to the voice of "Mom". The specific target voice model needs to be flexibly determined based on the actual input situation of the voice data to be trained.

[0110] 105. Send the target speech model so that the terminal device stores the target speech model.

[0111] In this embodiment, after the server obtains the target voice model through step 104, it can also send the target voice model to the terminal device. After the terminal device receives the target voice model, it can store the target voice model, that is, the terminal device loads the target voice model into the data structure of the terminal device through the integrated TTS-SDK. When the terminal device needs to play according to the target voice, it can generate the target voice data according to the stored target voice model, thereby completing the voice playback.

[0112] In an embodiment of the present application, a method for model training is provided. Using the above method, since the target speech model is obtained by training the speech model to be trained based on the predicted speech data set and the speech data set to be trained, the terminal device can obtain the corresponding target speech model from the speech data to be trained provided by the user, thereby improving the flexibility of speech model generation and meeting the user's customized needs for sound. Secondly, the possibility of speech model selection when the user plays voice is improved, thereby improving the flexibility of voice playback, and then improving the user's voice playback experience and flexibility.

[0113] Optionally, in the above Figure 4 On the basis of the corresponding embodiment, in an optional embodiment of the model training method provided in the embodiment of the present application, obtaining a speech data set to be trained specifically includes the following steps:

[0114] Get voice service request;

[0115] In response to the voice service request, a voice data collection instruction is sent to the terminal device, so that the terminal device responds to the voice data collection instruction and obtains the voice data to be detected, wherein the voice data to be detected is obtained based on the voice training text, and the voice training text is obtained according to the voice data collection instruction;

[0116] When the speech data to be detected meets the speech training qualification condition, the speech data to be trained is determined;

[0117] Based on the speech data to be trained, a speech data set to be trained is generated.

[0118] In this embodiment, a method for obtaining a set of speech data to be trained is provided. When a user needs to train a speech model, an operation for a speech service can be performed on a terminal device, including but not limited to single-clicking a speech training interface, double-clicking a speech training interface, and sliding a speech training interface. It should be understood that in actual applications, the operation can also be controlled by voice and gesture. For example, for voice control, the operation can be for a user to trigger a speech service through a speech input device (for example, a user says "customize my voice"). For gesture control, the operation can be for a user to trigger a gesture command through a corresponding gesture of the user. The operation is not specifically limited here.

[0119] Therefore, the server can obtain the voice service request, know that the terminal device needs to perform voice modeling, trigger the TVS service in the server, and generate a voice data collection instruction in response to the voice service request, thereby sending the voice data collection instruction to the terminal device. The terminal device can know from the voice data collection instruction that the server needs to obtain voice data for training. At this time, the terminal device needs to determine whether the voice data to be detected meets the noise qualification condition. If the noise qualification condition is met, the corresponding voice training text is obtained according to the voice data collection instruction. Then the user can perform voice input for the voice training text, and the terminal device obtains the voice input to be tested. The voice data to be detected is then sent to the server. After the server obtains the voice data to be detected, it transmits the voice data to be detected to the TTS customization module through the TVS-web interface, and then determines whether the voice data to be detected meets the qualified conditions for voice training. If so, it is determined that the voice data to be detected is the voice data to be trained. If not, the terminal device continues to obtain the corresponding voice data to be detected based on the user's voice input, and sends the voice data to be detected to the server, so that the server judges the voice data to be detected according to the aforementioned similar steps. After obtaining N voice data to be trained through the aforementioned similar steps, a set of voice data to be trained can be generated.

[0120] Specifically, the qualified condition for speech training is that the similarity between the speech data to be detected and the speech training text is greater than a preset threshold. In this embodiment, the similarity between the speech data to be detected and the speech training text should be greater than 95%. In practical applications, the qualified condition for speech training can also be that the similarity between the speech data to be detected and the speech training text should be greater than 98%, or the similarity between the speech data to be detected and the speech training text should be greater than 90%, etc. The specific qualified condition for speech training needs to be flexibly determined according to actual conditions and needs. For ease of understanding, the qualified condition for speech training is that the similarity between the speech data to be detected and the speech training text is greater than 95% as an example for explanation. The speech training text is "Follow me and jump together". If the text data obtained after semantic recognition of the speech data to be detected input by the user through voice is "Follow me and jump together", then it can be said that the similarity between the speech data to be detected and the speech training text is 100%, that is, the qualified condition for speech training is met. Secondly, the voice training text is "Eight hundred standard soldiers ran to the north slope, and the artillery soldiers on the north slope ran side by side", and the text data obtained after semantic recognition of the voice data to be tested input by the user through voice is "Eight hundred standard soldiers ran to the north slope, and the standard soldiers on the north slope did not need to run". Therefore, the similarity between the voice data to be tested and the voice training text is less than 95%, which does not meet the qualified conditions for voice training.

[0121] For easier understanding, see Figure 5 , Figure 5 This is another flow chart of the method for model training in an embodiment of the present application. As shown in the figure, in step A1, the terminal device sends a voice service request to the server. In step A2, after the server receives the voice service request, it responds to the voice service request and sends a voice data collection instruction to the terminal device. In step A3, after the terminal device receives the voice data collection instruction, it responds to the voice data collection instruction to first determine whether the voice data to be detected meets the noise qualification condition. If it meets the condition, a voice training text is obtained according to the voice data collection instruction. Then, the user performs voice input based on the voice training text to obtain the voice data to be detected, so that the terminal device obtains the voice data to be detected. In step A4, the acquired voice data to be detected is sent to the server. In step A5, when the server determines that the voice data to be detected meets the voice training qualification condition, it determines that the voice data to be detected is the voice data to be trained. In step A6, steps A1 to A5 are then repeated to generate a set of voice data to be trained based on the determined voice data to be trained.

[0122] It should be understood that the above examples are only used to understand the present solution. In actual applications, the speech training text needs to be determined based on the actual needs and design of the speech model, while the speech data to be tested is determined based on the user's voice input. Therefore, whether the speech training qualification conditions are met needs to be flexibly determined based on the speech training text and the actual situation of the speech data to be tested.

[0123] In an embodiment of the present application, a method for obtaining a set of speech data to be trained is provided. By adopting the above-mentioned method, the speech data to be detected is obtained through a speech data collection instruction, and the speech data to be detected that meets the speech training qualification conditions is determined as the speech data to be trained. Thus, a set of speech data to be trained including at least one speech data to be trained can be generated, thereby providing input data for the speech model to be trained, thereby improving the feasibility of the present solution.

[0124] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment of the model training method provided in the embodiments of the present application, based on the predicted speech data set and the speech data set to be trained, the speech model to be trained is trained to obtain the target speech model, which specifically includes the following steps:

[0125] Based on the predicted speech data set and the speech data set to be trained, updating the model parameters of the speech model to be trained according to the target loss function;

[0126] If the target loss function reaches convergence, the target speech model is generated according to the model parameters.

[0127] In this embodiment, a method for training a target speech model is provided. Since the server needs to train the model parameters of the speech model to be trained with the speech data set to be trained as the target, after obtaining a predicted speech data in the predicted speech data set, the server can obtain a speech data to be trained corresponding to the aforementioned predicted speech data from the speech data set to be trained, and then generate the value of the target loss function according to the predicted speech data and the corresponding speech data to be trained, and judge whether the target loss function reaches the convergence condition according to the value of the target loss function. If the convergence condition is not reached, the model parameters of the speech model to be trained are updated using the value of the target loss function. It can be seen that each time the speech model to be trained generates a predicted speech data, the server can perform the above operation once until the target loss function reaches the convergence condition, and then generate the target speech model according to the model parameters obtained after the model parameters are updated for the last time.

[0128] Among them, the target loss function can be the sum of the difference between the predicted speech data and the corresponding speech data to be trained, or the sum of the absolute values ​​of the difference between the predicted speech data and the corresponding speech data to be trained, or the square of the sum of the difference between the predicted speech data and the corresponding speech data to be trained, or other forms of target loss functions, etc. The selection of the specific target loss function can be determined according to the actual situation, and is not limited here. The convergence condition of the target loss function can be that the value of the target loss function is less than or equal to the first preset threshold. As an example, the value of the first preset threshold can be 0.005, 0.01, 0.02 or other values ​​close to 0; it can also be that the difference between two adjacent values ​​of the target loss function is less than or equal to the second preset threshold. The value of the second threshold can be the same as or different from the value of the first threshold. As an example, the value of the second preset threshold can be 0.005, 0.01, 0.02 or other values ​​close to 0, etc. The server can also adopt other convergence conditions, etc., which are not limited here.

[0129] In an embodiment of the present application, a method for training a target speech model is provided. Based on a predicted speech data set and a speech data set to be trained, the model parameters of the speech model to be trained are updated according to a target loss function. If the target loss function reaches convergence, the target speech model is generated according to the model parameters. By adopting the above method, a specific implementation method for the server to update the parameters of the speech model to be trained is provided, and the target speech model is generated thereby, thereby improving the feasibility of the present solution.

[0130] Optionally, in the above Figure 4 On the basis of the corresponding embodiment, in another optional embodiment of the method for model training provided in the embodiment of the present application, the method for model training further includes the following steps:

[0131] Obtain a voice playback request, wherein the voice playback request carries the voice data to be parsed;

[0132] Acquire service content text data according to the voice data to be parsed, wherein the service content text data corresponds to the voice data to be parsed;

[0133] Acquire toneless voice data according to the service content text data, wherein the toneless voice data is obtained by performing voice synthesis on the service content text data;

[0134] The timbre-free speech data is sent to the terminal device, so that the terminal device generates target speech data according to the timbre-free speech data and the target speech model, and plays the target speech data.

[0135] In this embodiment, when the user needs to play voice, the terminal device can detect the operation for voice playback, and input the voice data to be parsed through voice, thereby generating a voice playback request carrying the voice data to be parsed, and sending the voice playback request to the server. The server thereby obtains the voice playback request, and performs voice recognition on the voice data to be parsed carried in the voice playback request. The specific voice recognition can be semantic-to-text and semantic understanding, etc., and then the user's intention and needs are determined through voice recognition, thereby determining the service content text data, and then performing voice synthesis on the service content text data to complete the conversion of text data to voice data, thereby obtaining timbre-free voice data. The server sends a TTS instruction to the terminal, and the TTS instruction carries timbre-free voice data. Therefore, after the terminal obtains the TTS instruction carrying timbre-free voice data, it can generate target voice data based on the timbre-free voice data and the stored target voice model, and play the target voice data, thereby completing the voice playback.

[0136] For example, the voice data to be parsed carried in the voice playback request is "I want to listen to the ugly duckling" as an example for introduction. Figure 6 , Figure 6 This is a flow chart of the method for voice playback in the embodiment of the present application. As shown in the figure, in step F1, the user initiates a voice playback request to the terminal device according to the demand, and the voice data to be parsed carried in the voice playback request is "I want to listen to the ugly duckling". In step F2, the terminal device sends a voice playback request to the server. In step F3, the TVS service in the server can determine that the user's intention is to listen to a story, and the content he wants to listen to is "the ugly duckling story" after the process of speech recognition such as semantic conversion to text and semantic understanding. Therefore, a request is initiated to the skill service in the TVS service, specifically a request is initiated to the children's story content service, and then the skill service can feedback the children's story content service. In step F4, the server converts the service content text data "The Ugly Duckling Story" from text to sound, thereby obtaining timbre-free voice data, which is timbre-free voice data with voice content of the fairy tale "The Ugly Duckling" without pitch timbre, and then in step F5, the server sends the timbre-free voice data to the terminal device, and in step F6, the terminal device generates target voice data according to the timbre-free voice data and the target voice model, and plays the target voice data in step F7, that is, playing the story content of "The Ugly Duckling" through the target voice, for example, "Once upon a time, there was an ugly duckling..." It should be understood that the above examples are only used to understand this solution. In actual applications, the specific timbre-free voice data needs to be flexibly determined according to the actual situation of the voice data to be analyzed.

[0137] In an embodiment of the present application, a method for playing voice data based on a target voice model is provided, wherein the server obtains service content text data based on the voice data to be parsed carried in the voice playback request, and sends timbre-free voice data obtained based on the service content text data to the terminal device. By adopting the above method, the terminal device can generate target voice data based on the timbre-free voice data and the target voice model, and play the target voice data, thereby improving the feasibility of the present solution.

[0138] The following is an introduction to the method of voice playback in this application. Figure 7 , Figure 7 This is a schematic diagram of an embodiment of a method for voice playback in an embodiment of the present application. As shown in the figure, an embodiment of a method for voice playback in an embodiment of the present application includes:

[0139] 201. Detecting an operation for voice playback, sending a voice playback request, so that the server obtains service content text data based on the voice data to be parsed, and obtains timbre-free voice data according to the service content text data, wherein the voice playback request carries the voice data to be parsed, the service content text data is obtained according to the voice data to be parsed, and the timbre-free voice data is obtained by performing voice synthesis on the service content text data;

[0140] In this embodiment, when the user needs to play voice, the terminal device can detect the operation for voice playback, and perform voice input through the voice input interface to obtain the voice data to be parsed, and generate a voice playback request carrying the voice data to be parsed. Then the terminal device can send a voice playback request to the server through TVS-SDK, so that the server's TVS voice service performs voice recognition on the voice data to be parsed carried in the voice playback request to obtain service content text data, and then performs voice synthesis on the service content text data to obtain timbre-free voice data.

[0141] Specifically, the operation for voice playback includes but is not limited to single-clicking the voice training interface, double-clicking the voice training interface, and sliding the voice training interface. For ease of understanding, the terminal device is a mobile phone, and the operation for voice playback is single-clicking the voice training interface as an example for introduction. Please refer to Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the operation for voice playback in the embodiment of the present application. As shown in the figure, B1 is used to indicate the voice playback interface, and B2 is used to indicate the voice input interface. Figure 8 The middle (A) figure is used to instruct the user to click the voice playback interface B1, and then jump to Figure 8 The interface shown in Figure (B) is as follows: Figure 8 The middle (B) figure is used to indicate the interface for the user to input voice through the voice input interface B2, for example, the user Figure 8 After clicking the voice input interface B2 in the interface shown in Figure (B), say "I want to listen to the ugly duckling" to the terminal device, then the terminal device obtains the voice data to be parsed "I want to listen to the ugly duckling". It should be understood that the above examples are only used to understand this solution. In actual applications, the operation can also be controlled by voice and gestures. For example, for voice control, the operation can be for the user to trigger a voice service through a voice input device (for example, the user says "voice play"). For gesture control, the operation can be for the user to trigger a gesture command through a corresponding gesture of the user. The operation is not specifically limited here. Secondly, in actual applications, the shape, size, position and corresponding text information of the voice playback interface and the voice input interface need to be flexibly determined according to actual conditions.

[0142] 202. Acquire voice data without timbre;

[0143] In this embodiment, after the server obtains the timbre-free voice data in step 201, it sends the timbre-free voice data to the terminal device, and the TVS-SDK integrated in the terminal device obtains the timbre-free voice data. For example, the voice data to be analyzed is "I want to listen to the ugly duckling". It can be seen from the above embodiment that the timbre-free voice data is the voice data with the voice content of the fairy tale "The Ugly Duckling" and the toneless timbre. If the voice data to be analyzed is "I want to listen to the ugly monster", similar to the above embodiment, the timbre-free voice data can be the voice data with the voice content of the song "The Ugly Monster". It should be understood that the above examples are only used to understand this solution. In actual applications, the specific voice data to be analyzed needs to be flexibly determined according to the actual situation of user needs, and the specific timbre-free voice data needs to be flexibly determined according to the actual situation of the voice data to be analyzed.

[0144] 203. Generate target speech data according to the timbre-free speech data and the target speech model, wherein the target speech model is obtained by training the speech model to be trained based on the predicted speech data set and the speech data set to be trained, and the predicted speech data set is obtained by the server in response to the speech model training instruction, based on the speech data set to be trained, through the speech model to be trained;

[0145] In this embodiment, after the TVS-SDK integrated in the terminal device obtains the timbre-free voice data, it transmits the timbre-free voice data to the TTS-SDK. The user can select the target voice model at this time. It can be seen from the above embodiments that the target voice model is obtained by training the voice model to be trained based on the predicted voice data set and the voice data set to be trained. The predicted voice data set is obtained by the server in response to the voice model training instruction, based on the voice data set to be trained, and through the voice model to be trained, so it will not be repeated here.

[0146] Exemplarily, if the user uses the voice of "Dad" as the voice data to be trained, then the target voice model can be the voice model corresponding to the voice of "Dad". If the user uses the voice of "Mom" as the voice data to be trained, then the target voice model can be the voice model corresponding to the voice of "Mom". If the target voice model selected by the user is the voice model corresponding to the voice of "Mom", then TTS-SDK uses the timbre-free voice data as the input of the target voice model, and the target voice model outputs the target voice data, that is, the voice data corresponding to the timbre and tone of "Mom". It should be understood that the aforementioned examples are only used to understand this solution. In actual applications, the specific target voice data needs to be flexibly determined based on the actual situation of the timbre-free voice data and the target voice model selected by the user.

[0147] 204. Play the target voice data.

[0148] In this embodiment, after the terminal device obtains the target voice data, it can play the target voice data. For ease of understanding, the target voice model is a voice model corresponding to the sound of "mom" as an example for explanation. Fig. 9 , Fig. 9 This is another embodiment of the method for voice playback in the embodiment of the present application. As shown in the figure, C1 is used to indicate the voice input interface. After the user clicks the voice input interface C1, he says to the terminal device "I want to listen to the ugly duckling", then the terminal device obtains the voice data to be parsed "I want to listen to the ugly duckling". Through the steps of the aforementioned embodiment, the target voice data with the text content of the fairy tale "The Ugly Duckling" can be obtained. Therefore, the terminal device plays the target voice data "Once upon a time, there was an ugly duckling..." with the timbre and tone of "Mom". It should be understood that the aforementioned examples are only used to understand this solution. In actual applications, the specific target voice data needs to be flexibly determined according to the actual situation of the timbre-free voice data and the target voice model selected by the user.

[0149] In an embodiment of the present application, a method for voice playback is provided. By adopting the above method, a terminal device can generate target voice data based on timbre-free voice data and a target voice model. Since the target voice model is obtained from the voice data to be trained provided by the user, the flexibility of voice model generation is improved, thereby improving the flexibility of generating target voice data. Secondly, the generated target voice data is played. Due to the improvement in the flexibility of generating target voice data, the flexibility of voice playback can be improved.

[0150] Optionally, in the above Figure 7 On the basis of the corresponding embodiment, in an optional embodiment of the method for voice playback provided in the embodiment of the present application, the method for voice playback further includes the following steps:

[0151] Detecting operations for voice services and sending voice service requests;

[0152] When the server responds to the voice service request, it obtains a voice data collection instruction;

[0153] In response to the voice data collection instruction, when the voice data to be detected meets the noise qualification condition, the voice data to be detected is obtained, wherein the voice data to be detected is obtained based on the voice training text, and the voice training text is obtained according to the voice data collection instruction;

[0154] The voice data to be detected is sent so that the server generates a voice data set to be trained based on the voice data to be detected, where the voice data set to be trained includes N voice data to be trained, where N is a positive integer greater than or equal to 1.

[0155] In this embodiment, a method for sending voice data to be detected is provided. When a user needs to perform voice model training, an operation for voice service can be performed on the terminal device. The terminal device detects the operation for voice service and sends a voice service request to the server. The server can obtain the voice service request, learn that the terminal device needs to perform voice model training, trigger the TVS service in the server, and generate a voice data collection instruction in response to the voice service request, thereby sending the voice data collection instruction to the terminal device. The terminal device can learn from the voice data collection instruction that the server needs to obtain voice data for training, and then first determine whether the voice data to be detected meets the noise qualification condition. If it meets the condition, the corresponding voice training text is obtained according to the voice data collection instruction. Then the user can perform voice input for the voice training text, and the terminal device obtains the voice data to be detected input by the user.

[0156] Furthermore, the terminal device sends the voice data to be detected to the server. After the server obtains the voice data to be detected, it transmits the voice data to be detected to the TTS customization module through the TVS-web interface, and determines whether the voice data to be detected meets the qualified conditions for voice training. If so, it is determined that the voice data to be detected is the voice data to be trained. If not, the terminal device continues to send the voice data to be detected that meets the noise qualified conditions to the server. The server judges the voice data to be detected according to the above-mentioned similar steps. After obtaining N voice data to be trained through the above-mentioned similar steps, a set of voice data to be trained can be generated.

[0157] Specifically, the noise qualification condition is that the ambient noise is lower than a preset threshold value of decibels. In this embodiment, the noise qualification condition is that the ambient noise is lower than 45 decibels. It should be understood that in actual applications, if more accurate voice data to be detected is required, the noise qualification condition can also be that the ambient noise is lower than 40 decibels, or if faster acquisition of voice data to be detected is required, the noise qualification condition can also be that the ambient noise is lower than 50 decibels. The specific noise qualification condition is flexibly determined according to actual conditions and needs.

[0158] For easier understanding, see Fig.10 , Fig.10Another flow chart of the method for voice playback in the embodiment of the present application is shown in the figure. In step D1, the user performs an operation for the voice service, thereby generating a voice service request. In step D2, the terminal device detects the operation for the voice service and sends a voice service request to the server. In step D3, the server responds to the voice service request and sends a voice data collection instruction to the terminal device. In step D4, the terminal device responds to the voice data collection instruction to determine whether the voice data to be detected meets the noise qualification condition. If it meets the condition, step D5 is executed. In step D5, the terminal device obtains the voice training text according to the voice data collection instruction, and the user inputs the voice to be detected based on the voice training text. Detect voice data, in step D6, send the voice data to be detected to the server, in step D7, when the server determines that the voice data to be detected meets the qualified conditions for voice training, determine that the voice data to be detected is the voice data to be trained, and generate a voice data set to be trained based on the determined voice data to be trained, in step D8, when the number of voice data to be trained in the voice data set to be trained reaches the voice training number threshold, the terminal device can obtain the voice data collection completion status information, according to the voice data collection completion status information, it can be known that the voice data of the terminal device has been obtained, and the target voice model can be trained, so the server sends a voice model training instruction to the server. In step D9, after the server receives the voice model training instruction, it can be known that the terminal device needs to train the voice model to be trained, so it can respond to the voice model training instruction, perform model training to obtain the target voice model, in step D10 the server sends the target voice model to the terminal device, and in step D11 the terminal device stores the target voice model. In step D12, when the user needs to customize the voice playback, the terminal device can send a voice playback request to the server, in step D13, the server obtains the service content text data based on the voice data to be analyzed, in step D14, the server obtains the timbre-free voice data based on the service content text data, and then in step D15 the server sends the timbre-free voice data to the terminal device, in step D16, the terminal device generates the target voice data based on the timbre-free voice data and the target voice model, and in step D17, the terminal device plays the target voice data. The specific methods in the aforementioned steps have been introduced in the aforementioned embodiments, so they will not be repeated here. It should be understood that the aforementioned examples are only used to understand this solution. In actual applications, whether the specific voice data to be detected is to be sent to the server needs to be flexibly determined according to the situation of the user's voice input.

[0159] In an embodiment of the present application, a method for sending voice data to be detected is provided. By adopting the above-mentioned method, when the noise qualification condition is met, the voice data to be detected is obtained according to user demand, and the voice data to be detected is sent to a server, thereby improving the accuracy of the voice data to be detected, so as to facilitate the feature extraction of the voice data to be detected by the voice model to be trained, thereby improving the accuracy and efficiency of generating the target voice model, thereby improving the accuracy and efficiency of voice playback.

[0160] Optionally, in the above Figure 7 On the basis of the corresponding embodiment, in another optional embodiment of the method for voice playback provided in the embodiment of the present application, the method for voice playback further includes the following steps:

[0161] When N meets the speech training quantity threshold, a speech model training instruction is sent so that the server obtains the target speech model;

[0162] Obtain a target speech model;

[0163] Store the target speech model.

[0164] In this embodiment, a method for storing a target speech model is provided. After the terminal device sends the speech data to be detected to the server through the aforementioned steps, when the server determines that the number of speech data to be trained in the speech data set to be trained reaches the speech training number threshold, the terminal device can obtain the speech data collection completion status information. According to the speech data collection completion status information, it can be known that the speech data of the terminal device has been completed and the model training can be started. Therefore, a speech model training instruction is sent to the server. The server responds to the speech model training instruction, obtains a predicted speech data set based on the speech data set to be trained through the speech model to be trained, and trains the speech model to be trained based on the predicted speech data set and the speech data set to be trained to obtain the target speech model, and then sends the target speech model to the terminal device, and the terminal device stores the obtained target speech model. When the terminal device needs to play according to the target speech, the target speech data can be generated according to the stored target speech model, thereby completing the speech playback.

[0165] In an embodiment of the present application, a method for storing a target speech model is provided. By adopting the above method, the training of the target speech model can be started according to user needs, and the obtained target speech model can be stored thereby. When speech playback is required, target speech data can be generated by the target speech model, thereby completing speech playback and improving the feasibility of the present solution.

[0166] Optionally, in the above Figure 7 On the basis of the corresponding embodiment, in another optional embodiment of the method for voice playback provided in the embodiment of the present application, the method for voice playback further includes the following steps:

[0167] Acquiring voice data collection status information, wherein the voice data collection status information is used to indicate that the voice data to be detected is being acquired;

[0168] Acquiring voice data collection completion status information, wherein the voice data collection completion status information is used to indicate that the voice data has been acquired and that training of the target voice model can begin;

[0169] Obtaining target speech model training status information, wherein the target speech model training status information is used to indicate that the speech model to be trained is being trained;

[0170] Obtain target speech model completion status information, wherein the target speech model completion status information is used to indicate that the target speech model has completed training.

[0171] In this embodiment, the terminal device can also obtain different status information, and know the status of model training based on different status information. Specifically, the terminal device can obtain voice data collection status information, voice data collection completion status information, target voice model training status information and target voice model completion status information. Among them, the terminal device can know that the voice data to be detected is being obtained through the voice data collection status information, and the terminal device can know that the voice data has been completed through the voice data collection completion status information, that is, N meets the voice training quantity threshold, and the target voice model can be trained. Therefore, the terminal device sends a voice model training instruction to the server based on the voice data collection completion status information. Further, the terminal device can know that the server is training the voice model to be trained through the target voice model training status information, and the terminal device can know that the server target voice model has been trained through the target voice model completion status information, that is, the server has obtained the target voice model, and the target voice model is saved in the server's data, and then the server sends the target voice model to the terminal device, and the terminal device stores the trained target voice model. When the user requests TTV service through an operation for voice playback, the terminal device can use the stored target voice model and the timbre-free voice data to synthesize the target voice data.

[0172] For easier understanding, see Fig.11 , Fig.11This is a schematic diagram of another embodiment of the method for voice playback in the embodiment of the present application. As shown in the figure, in step E1, the terminal device continuously requests the server for detection model training status information. If the voice data to be detected is being obtained, in step E2, the terminal device can obtain the voice data collection status information. If the server generates a set of voice data to be trained including N voice data to be trained, and N meets the voice training quantity threshold, in step E2, the terminal device can obtain the voice data collection completion status information. At this time, the terminal device sends a voice model training instruction to the server, and the server starts model training. In step E3, the TTS customization service requests the model status information from the database. If the model training is completed to obtain the target voice model, step E4 is executed. If the model training is not completed, step E2 is executed. In step E2, the target voice model training status information is sent to the terminal device. In step E4, the offline training service sends the target voice model to the database. At this time, the TTS customization service can know that the model has been trained. Therefore, in step E5, the TTS customization service sends the target voice model completion status information to the message push service. At this time, in step E2, the message push service in the server sends the target voice model completion status information to the terminal device. Secondly, after the TVS-SDK of the terminal device receives the completion status information of the target speech model, it can also trigger the call of the TTS-SDK interface to check the model status. After determining that the target speech model has been generated through the target speech model completion status information, the model management module obtains the target speech model and saves it to the local model library. When the user chooses to use the target speech model, TTS-SDK loads the target speech model into the data structure for generating target speech data with the timbre-free speech data.

[0173] In an embodiment of the present application, another method for voice playback is provided. Using the above method, the terminal device can determine the progress of model training through different status information, so that the data required for the target voice model can be provided in a timely manner, and the target voice model can be obtained and stored in a timely manner after the training is completed, thereby improving the efficiency of subsequent generation of target voice data.

[0174] The model training device in this application is described in detail below. Fig.12 , Fig.12 This is a schematic diagram of an embodiment of a model training device in an embodiment of the present application. As shown in the figure, the model training device 30 includes:

[0175] The acquisition module 301 is used to acquire a speech data set to be trained, wherein the speech data set to be trained includes N speech data to be trained, where N is a positive integer greater than or equal to 1;

[0176] The acquisition module 301 is further used to acquire a speech model training instruction when N meets a speech training quantity threshold;

[0177] The acquisition module 301 is further used to respond to the speech model training instruction and, based on the speech data set to be trained, obtain a predicted speech data set through the speech model to be trained, wherein the predicted speech data set includes N predicted speech data, and the predicted speech data has a corresponding relationship with the speech data to be trained;

[0178] A training module 302 is used to train the speech model to be trained based on the predicted speech data set and the speech data set to be trained to obtain a target speech model;

[0179] The sending module 303 is used to send the target speech model so that the terminal device stores the target speech model.

[0180] Optionally, in the above Fig.12 On the basis of the corresponding embodiment, in another embodiment of the model training device 30 provided in the embodiment of the present application,

[0181] The acquisition module 301 is specifically used to acquire a voice service request;

[0182] In response to the voice service request, a voice data collection instruction is sent to the terminal device, so that the terminal device responds to the voice data collection instruction and obtains the voice data to be detected, wherein the voice data to be detected is obtained based on the voice training text, and the voice training text is obtained according to the voice data collection instruction;

[0183] When the speech data to be detected meets the speech training qualification condition, the speech data to be trained is determined;

[0184] Based on the speech data to be trained, a speech data set to be trained is generated.

[0185] Optionally, in the above Fig.12 On the basis of the corresponding embodiment, in another embodiment of the model training device 30 provided in the embodiment of the present application,

[0186] The training module 302 is specifically used to update the model parameters of the speech model to be trained according to the target loss function based on the predicted speech data set and the speech data set to be trained;

[0187] If the target loss function reaches convergence, the target speech model is generated according to the model parameters.

[0188] Optionally, in the above Fig.12 On the basis of the corresponding embodiment, in another embodiment of the model training device 30 provided in the embodiment of the present application,

[0189] The acquisition module 301 is also used to acquire a voice playback request, wherein the voice playback request carries the voice data to be parsed;

[0190] The acquisition module 301 is further used to acquire service content text data according to the voice data to be parsed, wherein the service content text data corresponds to the voice data to be parsed;

[0191] The acquisition module 301 is further used to acquire the timbre-free speech data according to the service content text data, wherein the timbre-free speech data is obtained by performing speech synthesis on the service content text data;

[0192] The sending module 303 is further used to send the timbre-free speech data to the terminal device, so that the terminal device generates target speech data according to the timbre-free speech data and the target speech model, and plays the target speech data.

[0193] The following is a detailed description of the voice playback device in this application. Fig.13 , Fig.13 This is a schematic diagram of an embodiment of a voice playback device in an embodiment of the present application. As shown in the figure, the voice playback device 40 includes:

[0194] The sending module 401 is used to detect an operation for voice playback and send a voice playback request, so that the server obtains service content text data based on the voice data to be parsed, and obtains timbre-free voice data according to the service content text data, wherein the voice playback request carries the voice data to be parsed, the service content text data is obtained according to the voice data to be parsed, and the timbre-free voice data is obtained by performing voice synthesis on the service content text data;

[0195] An acquisition module 402 is used to acquire voice data without timbre;

[0196] A generating module 403 is used to generate target speech data according to the timbre-free speech data and the target speech model, wherein the target speech model is obtained by training the speech model to be trained based on the predicted speech data set and the speech data set to be trained, and the predicted speech data set is obtained by the server in response to the speech model training instruction, based on the speech data set to be trained, through the speech model to be trained;

[0197] The playing module 404 is used to play the target voice data.

[0198] Optionally, in the above Fig.13 On the basis of the corresponding embodiment, in another embodiment of the voice playback device 40 provided in the embodiment of the present application,

[0199] The sending module 401 is further used to detect the operation for the voice service and send a voice service request;

[0200] The acquisition module 402 is further used to acquire a voice data collection instruction when the server responds to the voice service request;

[0201] The acquisition module 402 is further configured to respond to the voice data collection instruction, and when the voice data to be detected meets the noise qualification condition, acquire the voice data to be detected, wherein the voice data to be detected is obtained based on the voice training text, and the voice training text is obtained according to the voice data collection instruction;

[0202] The sending module 401 is further used to send the speech data to be detected, so that the server generates a speech data set to be trained based on the speech data to be detected, and the speech data set to be trained includes N speech data to be trained, where N is a positive integer greater than or equal to 1.

[0203] Optionally, in the above Fig.13 On the basis of the corresponding embodiment, in another embodiment of the voice playback device 40 provided in the embodiment of the present application, the voice playback device 40 further includes a storage module 405,

[0204] The sending module 401 is further used to send a speech model training instruction when N meets the speech training quantity threshold, so that the server obtains the target speech model;

[0205] The acquisition module 402 is also used to acquire a target speech model;

[0206] The storage module 405 is used to store the target speech model.

[0207] Optionally, in the above Fig.13 On the basis of the corresponding embodiment, in another embodiment of the voice playback device 40 provided in the embodiment of the present application,

[0208] The acquisition module 402 is further used to acquire voice data collection status information, wherein the voice data collection status information is used to indicate that the voice data to be detected is being acquired;

[0209] The acquisition module 402 is further used to acquire speech data collection completion status information, wherein the speech data collection completion status information is used to indicate that the speech data has been acquired and the training of the target speech model can be started;

[0210] The acquisition module 402 is further used to acquire the target speech model training status information, wherein the target speech model training status information is used to indicate that the speech model to be trained is being trained;

[0211] The acquisition module 402 is further used to acquire target speech model completion status information, wherein the target speech model completion status information is used to indicate that the target speech model has completed training.

[0212] The present application also provides a server. Fig.14 , Fig.14This is a structural diagram of a server in an embodiment of the present application. As shown in the figure, the server 500 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 522 (for example, one or more processors) and a memory 532, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 542 or data 544. Among them, the memory 532 and the storage medium 530 can be short-term storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 522 can be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the server 500.

[0213] The server 500 may also include one or more power supplies 526, one or more wired or wireless network interfaces 550, one or more input and output interfaces 558, and / or one or more operating systems 541, such as Windows Server 2003, ... TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM etc.

[0214] The steps performed by the server in the above embodiment can be based on the Fig.14 The server structure shown.

[0215] The present application also provides a terminal device, on which a client is deployed, such as Fig.15 For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal device may include but is not limited to any terminal device such as a smart phone, a tablet computer, a laptop computer, a PDA, a personal computer, a smart watch, a TV, etc., taking the terminal device as a mobile phone as an example:

[0216] Fig.15 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Fig.15As shown in the figure, the mobile phone includes: a radio frequency (RF) circuit 610, a memory 620, an input unit 630, a display unit 640, a sensor 630, a voice circuit 660, a wireless fidelity (WiFi) module 670, a processor 680, and a power supply 690. Those skilled in the art can understand that Fig.15 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0217] Combine the following Fig.15 A detailed introduction to the various components of the mobile phone:

[0218] The RF circuit 610 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 680 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 610 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 610 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0219] The memory 620 can be used to store software programs and modules. The processor 680 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 620. The memory 620 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as voice data, phone book, etc.), etc. In addition, the memory 620 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0220] The input unit 630 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 630 may include a touch panel 631 and other input devices 632. The touch panel 631, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 631 or near the touch panel 631 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 631 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 680, and can receive and execute commands sent by the processor 680. In addition, the touch panel 631 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 631, the input unit 630 may also include other input devices 632. Specifically, the other input devices 632 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0221] The display unit 640 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 640 may include a display panel 641. Optionally, the display panel 641 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 631 may cover the display panel 641. When the touch panel 631 detects a touch operation on or near it, it is transmitted to the processor 680 to determine the type of touch event. Subsequently, the processor 680 provides a corresponding visual output on the display panel 641 according to the type of touch event. Although in Fig.15 In the embodiment, the touch panel 631 and the display panel 641 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 631 and the display panel 641 can be integrated to realize the input and output functions of the mobile phone.

[0222] The mobile phone may also include at least one sensor 630, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 641 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 641 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0223] The voice circuit 660, the speaker 661, and the microphone 662 can provide a voice interface between the user and the mobile phone. The voice circuit 660 can transmit the electrical signal converted from the received voice data to the speaker 661, which is converted into a sound signal for output; on the other hand, the microphone 662 converts the collected sound signal into an electrical signal, which is received by the voice circuit 660 and converted into voice data, and then the voice data is output to the processor 680 for processing, and then sent to another mobile phone through the RF circuit 610, or the voice data is output to the memory 620 for further processing.

[0224] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 670. It provides users with wireless broadband Internet access. Fig.15A WiFi module 670 is shown, but it is understandable that it is not an essential component of the mobile phone.

[0225] The processor 680 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 620, and calling data stored in the memory 620. Optionally, the processor 680 may include one or more processing units; preferably, the processor 680 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 680.

[0226] The mobile phone also includes a power supply 690 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 680 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.

[0227] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0228] In the embodiment of the present application, the processor 680 included in the terminal device can perform the functions in the aforementioned embodiment, which will not be repeated here.

[0229] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the steps executed by the terminal device in the method described in the aforementioned embodiment, or enables the computer to execute the steps executed by the server in the method described in the aforementioned embodiment.

[0230] A computer program product including a program is also provided in an embodiment of the present application. When the program is run on a computer, the computer executes the steps executed by the terminal device in the method described in the aforementioned embodiment, or the computer executes the steps executed by the server in the method described in the aforementioned embodiment.

[0231] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0232] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0233] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0235] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0236] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A model training method, characterized in that: include: Acquire a speech data set to be trained, wherein the speech data set to be trained includes N speech data to be trained, where N is a positive integer greater than or equal to 1; When N meets the speech training quantity threshold, obtain the speech model training instruction; In response to the speech model training instruction, the speech data set to be trained is used as an input of the speech model to be trained, and a predicted speech data set is obtained through the speech model to be trained, wherein the predicted speech data set includes N predicted speech data, and the predicted speech data has a corresponding relationship with the speech data to be trained; Based on the predicted speech data set and the speech data set to be trained, updating the model parameters of the speech model to be trained according to a target loss function; the target loss function is determined according to the difference between the predicted speech data and the speech data to be trained; If the target loss function reaches convergence, generating a target speech model according to the model parameters; Sending the target speech model so that the terminal device stores the target speech model; Obtaining a voice playback request, wherein the voice playback request carries voice data to be parsed; Acquiring service content text data according to the voice data to be parsed, wherein the service content text data corresponds to the voice data to be parsed; Acquire timbre-free speech data according to the service content text data, wherein the timbre-free speech data is obtained by performing speech synthesis on the service content text data; The timbre-free speech data is sent to a terminal device, so that the terminal device uses the timbre-free speech data as an input of the target speech model, outputs the target speech data through the target speech model, and plays the target speech data.

2. The method according to claim 1, characterized in that The step of obtaining a speech data set to be trained includes: Get voice service request; In response to the voice service request, sending a voice data collection instruction to the terminal device, so that the terminal device obtains the voice data to be detected in response to the voice data collection instruction, wherein the voice data to be detected is obtained based on a voice training text, and the voice training text is obtained according to the voice data collection instruction; When the speech data to be detected meets the speech training qualification condition, determining the speech data to be trained; Based on the speech data to be trained, the speech data set to be trained is generated.

3. A method for voice playback, characterized in that: include: Detecting an operation for voice playback, sending a voice playback request, so that the server obtains service content text data based on the voice data to be parsed, and obtains timbre-free voice data according to the service content text data, wherein the voice playback request carries the voice data to be parsed, the service content text data is obtained according to the voice data to be parsed, and the timbre-free voice data is obtained by performing voice synthesis on the service content text data; Acquiring the timbre-free speech data; The timbre-free speech data is used as the input of the target speech model, and the target speech data is output through the target speech model, wherein the target speech model is obtained by training the speech model to be trained based on the predicted speech data set and the speech data set to be trained, and the predicted speech data set is obtained by the speech model to be trained by the server in response to the speech model training instruction, using the speech data set to be trained as the input of the speech model to be trained; the training of the speech model to be trained based on the predicted speech data set and the speech data set to be trained includes: updating the model parameters of the speech model to be trained according to the target loss function based on the predicted speech data set and the speech data set to be trained; the target loss function is determined according to the difference between the predicted speech data and the speech data to be trained; if the target loss function reaches convergence, the target speech model is generated according to the model parameters; Play the target voice data.

4. The method according to claim 3, characterized in that The method further comprises: Detecting operations for voice services and sending voice service requests; When the server responds to the voice service request, obtaining a voice data collection instruction; In response to the voice data collection instruction, when the voice data to be detected meets the noise qualification condition, the voice data to be detected is obtained, wherein the voice data to be detected is obtained based on a voice training text, and the voice training text is obtained according to the voice data collection instruction; The speech data to be detected is sent so that the server generates a speech data set to be trained based on the speech data to be detected, wherein the speech data set to be trained includes N speech data to be trained, where N is a positive integer greater than or equal to 1.

5. The method according to claim 4, characterized in that The method further comprises: When N meets the speech training quantity threshold, a speech model training instruction is sent so that the server obtains the target speech model; Obtain a target speech model; The target speech model is stored.

6. The method according to any one of claims 3 to 5, characterized in that The method further comprises: Acquiring voice data collection status information, wherein the voice data collection status information is used to indicate that the voice data to be detected is being acquired; Acquiring speech data collection completion status information, wherein the speech data collection completion status information is used to indicate that the speech data has been acquired and that training of the target speech model can begin; Acquire target speech model training status information, wherein the target speech model training status information is used to indicate that the speech model to be trained is being trained; Obtain target speech model completion status information, wherein the target speech model completion status information is used to indicate that the target speech model has completed training.

7. A model training device, characterized in that: include: An acquisition module, used for acquiring a speech data set to be trained, wherein the speech data set to be trained includes N speech data to be trained, where N is a positive integer greater than or equal to 1; The acquisition module is further used to acquire a speech model training instruction when N meets a speech training quantity threshold; The acquisition module is further used to respond to the speech model training instruction, use the speech data set to be trained as the input of the speech model to be trained, and obtain a predicted speech data set through the speech model to be trained, wherein the predicted speech data set includes N predicted speech data, and the predicted speech data has a corresponding relationship with the speech data to be trained; A training module, configured to update the model parameters of the speech model to be trained according to a target loss function based on the predicted speech data set and the speech data set to be trained; the target loss function is determined according to the difference between the predicted speech data and the speech data to be trained; The training module is further configured to generate a target speech model according to the model parameters if the target loss function reaches convergence; A sending module, used for sending the target speech model so that the terminal device stores the target speech model; The acquisition module is further used to acquire a voice playback request, wherein the voice playback request carries the voice data to be parsed; The acquisition module is further used to acquire service content text data according to the voice data to be parsed, wherein the service content text data corresponds to the voice data to be parsed; The acquisition module is further used to acquire toneless voice data according to the service content text data, wherein the toneless voice data is obtained by performing voice synthesis on the service content text data; The sending module is also used to send the timbre-free speech data to the terminal device, so that the terminal device uses the timbre-free speech data as the input of the target speech model, outputs the target speech data through the target speech model, and plays the target speech data.

8. The device according to claim 7, characterized in that The acquisition module is specifically used for: Get voice service request; In response to the voice service request, sending a voice data collection instruction to the terminal device, so that the terminal device obtains the voice data to be detected in response to the voice data collection instruction, wherein the voice data to be detected is obtained based on a voice training text, and the voice training text is obtained according to the voice data collection instruction; When the speech data to be detected meets the speech training qualification condition, determining the speech data to be trained; Based on the speech data to be trained, the speech data set to be trained is generated.

9. A device for voice playback, characterized in that: include: A sending module, used for detecting an operation for voice playback, sending a voice playback request, so that the server obtains service content text data based on the voice data to be parsed, and obtains timbre-free voice data according to the service content text data, wherein the voice playback request carries the voice data to be parsed, the service content text data is obtained according to the voice data to be parsed, and the timbre-free voice data is obtained by performing voice synthesis on the service content text data; An acquisition module, used for acquiring the toneless voice data; A generation module, used for taking the timbre-free speech data as the input of a target speech model, and outputting target speech data through the target speech model, wherein the target speech model is obtained by training a speech model to be trained based on a predicted speech data set and a speech data set to be trained, and the predicted speech data set is obtained by the speech model to be trained by the server in response to a speech model training instruction, taking the speech data set to be trained as the input of the speech model to be trained; the training of the speech model to be trained based on the predicted speech data set and the speech data set to be trained comprises: updating the model parameters of the speech model to be trained according to a target loss function based on the predicted speech data set and the speech data set to be trained; the target loss function is determined according to the difference between the predicted speech data and the speech data to be trained; if the target loss function reaches convergence, generating the target speech model according to the model parameters; A playing module is used to play the target voice data.

10. The device according to claim 9, characterized in that The sending module is further used to detect an operation for a voice service and send a voice service request; The acquisition module is further configured to acquire a voice data collection instruction when the server responds to the voice service request; The acquisition module is further used to respond to the voice data collection instruction, and when the voice data to be detected meets the noise qualification condition, acquire the voice data to be detected, wherein the voice data to be detected is obtained based on the voice training text, and the voice training text is obtained according to the voice data collection instruction; The sending module is also used to send the voice data to be detected, so that the server generates a voice data set to be trained based on the voice data to be detected, and the voice data set to be trained includes N voice data to be trained, where N is a positive integer greater than or equal to 1.

11. The device according to claim 10, characterized in that The device also includes: a storage module; The sending module is further configured to send a speech model training instruction when N meets a speech training quantity threshold, so that the server obtains a target speech model; The acquisition module is also used to acquire the target speech model; The storage module is used to store the target speech model.

12. The device according to any one of claims 9 to 11, characterized in that The acquisition module is further used to acquire voice data collection status information, wherein the voice data collection status information is used to indicate that the voice data to be detected is being acquired; The acquisition module is further used to acquire speech data collection completion status information, wherein the speech data collection completion status information is used to indicate that the speech data has been acquired and the target speech model can be trained; The acquisition module is further used to acquire target speech model training status information, wherein the target speech model training status information is used to indicate that the speech model to be trained is being trained; The acquisition module is further used to acquire completion status information of the target speech model, wherein the completion status information of the target speech model is used to indicate that the training of the target speech model has been completed.

13. A server, characterized in that: include: memories, transceivers, processors, and bus systems; Wherein, the memory is used to store programs; The processor is used to execute the program in the memory, and the processor is used to execute the method according to any one of claims 1 to 2 according to the instructions in the program; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

14. A terminal device, characterized in that: include: memories, transceivers, processors, and bus systems; Wherein, the memory is used to store programs; The processor is used to execute the program in the memory, and the processor is used to execute the method according to any one of claims 3 to 6 according to the instructions in the program; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the method according to any one of claims 1 to 2, or the method according to any one of claims 3 to 6.

16. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of any one of claims 1 to 2, or the method of any one of claims 3 to 6.

Images