Text processing method and device, apparatus and storage medium for training model thereof

Abstract

The present disclosure provides a text processing method and model training method, device and storage medium, relates to the technical field of computers, in particular to the field of artificial intelligence such as deep learning and speech synthesis. The text processing method comprises: performing regularization processing on a text to obtain a regularized text; extracting a representation feature of the regularized text; and performing a plurality of tasks in parallel based on the representation feature to obtain a task result of each task in the plurality of tasks. The present disclosure can improve the text processing effect.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to artificial intelligence fields such as deep learning and speech synthesis, and more particularly to a text processing and model training method, apparatus, device and storage medium. Background Art

[0002] Speech synthesis, also known as text to speech (TTS), refers to the technology of converting text into speech.

[0003] The speech synthesis system mainly includes a text front-end module, an acoustic model and a vocoder. The text front-end module is used to convert text into linguistic features or phonemes. The acoustic model is used to convert the linguistic features or phonemes output by the text front-end module into acoustic features, such as Mel spectrum features. The vocoder is used to convert the acoustic features output by the acoustic model into speech. Summary of the Invention

[0004] The present disclosure provides a text processing and model training method, apparatus, device and storage medium.

[0005] According to one aspect of the present disclosure, a text processing method is provided, comprising: performing regularization processing on a text to obtain regularized text; extracting representation features of the regularized text; and executing multiple tasks in parallel based on the representation features to obtain task results of each of the multiple tasks.

[0006] According to another aspect of the present disclosure, a training method for a text processing model is provided, wherein the text processing model includes: a feature extraction model and multiple task processing models, and the method includes: using the feature extraction model to extract features of an input training text to output representation features of the training text; using each task processing model among the multiple task processing models to process the representation features to output prediction results of each task; constructing a loss function based on the prediction results and label data of the training text; and adjusting model parameters of the feature extraction model and model parameters of at least one task processing model among the multiple task processing models based on the loss function.

[0007] According to another aspect of the present disclosure, a text processing device is provided, including: a regularization module for performing regularization processing on a text to obtain a regularized text; a feature extraction module for extracting representation features of the regularized text; and a task execution module for executing a plurality of tasks in parallel based on the representation features to obtain a task result of each of the plurality of tasks.

[0008] According to another aspect of the present disclosure, there is provided a device for training a text processing model, the text processing model comprising a feature extraction model and a plurality of task processing models, the device comprising: a feature extraction module configured to perform feature extraction on an input training text using the feature extraction model to output a representation feature of the training text; a task execution module configured to perform processing on the representation feature using each of the plurality of task processing models to output a prediction result of each task; a construction module configured to construct a loss function based on the prediction result and label data of the training text; and an adjustment module configured to adjust model parameters of the feature extraction model and model parameters of at least one of the plurality of task processing models based on the loss function.

[0009] According to another aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of the above aspects.

[0010] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method of any one of the above aspects.

[0011] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of any one of the above aspects.

[0012] According to the technical solutions of the present disclosure, the text processing effect can be improved.

[0013] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings are used to better understand the present solution and do not limit the present disclosure. Among them:

[0015] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0016] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0017] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0018] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0019] Figure 5 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0020] Figure 6 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0021] Figure 7 is a schematic diagram according to a seventh embodiment of the present disclosure;

[0022] Figure 8 is a schematic diagram according to an eighth embodiment of the present disclosure;

[0023] Figure 9 is a schematic diagram according to a ninth embodiment of the present disclosure;

[0024] Figure 10 Schematic diagram of an electronic device used to implement the text processing method or text processing model training method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure. This embodiment provides a text processing method, including:

[0027] 101. Regularize the text to obtain regularized text.

[0028] 102. Extract representation features of the regularized text.

[0029] 103. Based on the representation features, execute multiple tasks in parallel to obtain a task result of each of the multiple tasks.

[0030] The execution subject of the text processing method provided in this embodiment can be referred to as a text processing device, which can be software, hardware, or a combination of software and hardware. The device can be located in an electronic device, and the electronic device can include: a terminal device or a server. The terminal device can include a mobile device (such as a mobile phone, a portable computer, etc.), a smart home device (such as a smart speaker, a smart TV, etc.), a smart wearable device (such as a smart watch, a smart bracelet, etc.), etc. The server can be a local server or a cloud server.

[0031] The text processing method provided in this embodiment can be specifically applied to the text front-end module of a Chinese speech synthesis system.

[0032] like Figure 2 As shown, the speech synthesis system 200 may include: a text front-end module 201 , an acoustic model 202 and a vocoder 203 .

[0033] The text front-end module 201 is used to convert text into linguistic features or phonemes, the acoustic model 202 is used to convert the linguistic features or phonemes output by the text front-end module into acoustic features, such as Mel spectrum features, and the vocoder 203 is used to convert the acoustic features output by the acoustic model into speech.

[0034] The text front-end module mainly includes the following modules:

[0035] Text regularization module, word segmentation (word segmentation can also include part-of-speech tagging) module, grapheme-to-phoneme (G2P) conversion module, and rhythm prediction module, wherein the grapheme-to-phoneme conversion module can include a polyphone prediction module and a tone change module.

[0036] Combined with the following examples, the functions of each module in the text front-end module are explained as follows:

[0037] Text: There are 112 211 universities in China.

[0038] Text regularization: There are a total of 112 211 universities in China

[0039] Participle: There are a total of 112 211 colleges and universities in the country

[0040] Word-to-phonetic conversion (note the pronunciation of "一" in this sentence):

[0041] quan2 guo2 yi2 gong4 you3 yi4 bai3 yi1 shi2 er4 suo3 er4 yao1yao1gao1 xiao4

[0042] Among them, when converting the pronunciation of a word, the initial consonant and the final vowel can be further separated, or the tone and the initial consonant and the final vowel can be separated.

[0043] Prosody prediction (prosody word #1, prosody phrase #2, intonation phrase #3, sentence #4):

[0044] There are a total of 2112 universities in China.

[0045] In related technologies, pure linguistic rule design can be used. However, this requires developers to have professional linguistic knowledge, is prone to errors, is extremely labor-intensive, and has limited cross-language applicability. Alternatively, a serial approach is used for word segmentation, character-to-phonetic conversion, etc. In this approach, each module (word segmentation, character-to-phonetic conversion, etc.) corresponds to a model. In this serial approach, the various models are interdependent, which poses problems in terms of accuracy and efficiency.

[0046] To at least partially address at least one of the above issues, in the disclosed embodiments, multiple tasks can be executed in parallel, where "multiple" refers to at least two tasks. Each task is a task that can be performed by a separate model in the text front-end, such as word segmentation, part-of-speech tagging, prosody prediction, polyphone prediction, and tone inflection.

[0047] Through the above-mentioned tasks such as word segmentation and part-of-speech tagging, it can be applied to the text front-end module.

[0048] Before executing multiple tasks in parallel, the representation features of the text can be extracted first, that is, the text can be converted from text form to vector form so that the vector can be processed subsequently using a deep learning model.

[0049] For example, see Figure 3 , assuming there are N tasks in total, where N is a positive integer greater than 1.

[0050] A regularization module may be used to perform regularization processing on the text to obtain regularized text, wherein the regularization module may be executed based on rules. For example, regularization rules may be pre-set and regularization processing may be performed based on the regularization rules.

[0051] A feature extraction module can be used to extract representation features of the regularized text, where the feature extraction module can be a pre-trained model, such as a bidirectional Transformer Encoder (Bidirectional Encoder Representations from Transformers, BERT) model, a knowledge enhanced semantic representation (Enhanced Representation from kNowledge IntEgration, ERNIE) model, and other pre-trained models.

[0052] A task processing module for each task can be used to process the representation features and output the task results of each task, such as word segmentation results, prosody prediction results, etc. The task processing module for each task can specifically be a classification network model, such as a fully connected (FC) network model.

[0053] In this embodiment, by extracting representational features from the regularized text and executing multiple tasks in parallel based on these features, the extraction and processing based on these features eliminates the need for purely linguistic design and avoids the problems associated with such design. Furthermore, by executing multiple tasks in parallel, interdependence between tasks can be avoided, compared to a serial approach, resulting in improvements in accuracy and efficiency. Therefore, this embodiment can enhance text processing effectiveness.

[0054] Figure 4 This embodiment provides a text processing method, and combines Figure 5 As shown in the structural diagram, the method of this embodiment includes:

[0055] 401. Regularize the text to obtain regularized text.

[0056] Among them, regularization processing can also be called text normalization (TN), which converts non-standard words (NSW) into spoken-form words (SFW).

[0057] The raw input to a speech synthesis system may contain a variety of complex input types, such as numbers, cardinal numbers, ordinal numbers, numerical ranges, dates, times, temperatures, fractions, decimals, percentages, and phone numbers. In a speech synthesis system, a text normalization module is first used to convert these complex inputs into plain text. This converted plain text is then used as input for subsequent modules, such as word segmentation, tone inflection, polyphone prediction, and prosody prediction.

[0058] Among them, the text regularization module can be completed in a pure rule-based manner. The example input and output are shown in Table 1:

[0059] Table 1

[0060]

[0061]

[0062] By regularizing the text, the text can be standardized to facilitate subsequent processing.

[0063] 402. Use a pre-trained language model to perform feature extraction on the input regularized text to output representation features of the regularized text.

[0064] The pre-trained language model can be a BERT model or an ERNIE model. Pre-trained language models are generally trained using large amounts of data, such as text corpora in the billions. Therefore, pre-trained language models have good performance in terms of accuracy and generalization.

[0065] By using a pre-trained language model to extract representation features of regularized text, representation features with better expressiveness can be obtained.

[0066] 403. Use a single deep learning model corresponding to each task in the multiple tasks to process the input representation features to output task results for each task.

[0067] Among them, the deep learning module can be a classification network model, such as an FC network model.

[0068] For one or some tasks, non-deep learning models can also be used for processing. For example, for word segmentation, Jieba word segmentation can be used.

[0069] However, in order to improve accuracy, deep learning models can be used for processing.

[0070] Furthermore, the deep learning model adopts a classification network model, which can simplify the implementation.

[0071] Take multiple tasks including: word segmentation-part-of-speech tagging (i.e. completing word segmentation and part-of-speech tagging), rhythm prediction, tone change, and polyphone prediction as an example. Figure 5 As shown in Figure 1, multiple tasks can adopt a parallel structure, and the input of each task in the multiple tasks is the representation feature output by the pre-trained model.

[0072] In this embodiment, by regularizing the text, extracting the representation features of the regularized text, and executing multiple tasks in parallel based on the representation features, a text front-end processing process based on rules and deep learning can be realized. Since it is not a purely linguistic design scheme, the problem that the traditional text front-end module based on pure rules requires the participation of linguistic experts in research and development and the difficulty of cross-language migration is avoided. The deep learning model of multiple tasks adopts a parallel structure, each model is independent, and only relies on a unified pre-trained language model, and the errors between the models are no longer transmitted; the representation features of the pre-trained language module (such as word vectors) are context-related expressions, and the training data of the pre-trained language model is almost unlimited; through the parallel processing of each model, the overall prediction speed is faster.

[0073] The above involves the application process of the model. For the training process of the model, please refer to the following embodiments.

[0074] Figure 6 6 is a schematic diagram according to the sixth embodiment of the present disclosure. This embodiment provides a method for training a text processing model, wherein the text processing model includes: a feature extraction model and multiple task processing models, and the method includes:

[0075] 601. Use the feature extraction model to perform feature extraction on the input training text to output representation features of the training text.

[0076] 602. Use each task processing model among the multiple task processing models to process the representation features to output a prediction result for each task.

[0077] 603. Construct a loss function based on the prediction result and the label data of the training text.

[0078] 604. Based on the loss function, adjust model parameters of the feature extraction model and model parameters of at least one task processing model among the multiple task processing models.

[0079] The text processing model trained in this embodiment can be applied to the text front-end module of the Chinese speech synthesis system.

[0080] like Figure 7 As shown, the text processing model 700 may include a feature extraction model 701 and multiple task processing models 702. Figure 7 The first to fourth task processing models are used to represent them respectively.

[0081] The multiple task processing models are used to perform at least two of the following tasks: at least two of word segmentation, part-of-speech tagging, prosody prediction, polyphone prediction, and tone change.

[0082] Through the above-mentioned tasks such as word segmentation and part-of-speech tagging, it can be applied to the text front-end module.

[0083] The above training text may be collected in advance, and may specifically be regularized text.

[0084] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0085] The feature extraction model may be a pre-trained prediction model; and / or each task processing model may be a classification network model.

[0086] The pre-trained language model can be a BERT model or an ERNIE model. Pre-trained language models are generally trained using large amounts of data, such as text corpora in the billions. Therefore, pre-trained language models have good performance in terms of accuracy and generalization.

[0087] By using a pre-trained language model to extract representation features of training text, representation features with better expressiveness can be obtained.

[0088] The deep learning model uses the FC network model, which can simplify the implementation.

[0089] During training, each task in multiple tasks can have a different data set. The labels of some data sets can be used for multiple tasks at the same time, and the labels of some data sets can only be used for one task. In the data preprocessing stage, all labels of all data sets of all tasks can be integrated and the data can be randomly sorted. During training, the model determines whether to use the data for the calculation of a subtask loss function based on the label of each data. For example, if a piece of data only contains the label of the rhythm prediction task, then the data is only used to calculate the rhythm prediction related loss function. If a piece of data contains labels for both polyphone prediction and rhythm prediction, then the data is used for the calculation of both polyphone prediction and rhythm prediction related labels.

[0090] Assume that there are 4 tasks in the multi-task system, which can be called the first to fourth tasks respectively. Then the loss function L = L1 + L2 + L3 + L4, where L1 to L4 are constructed based on the prediction results and label data of each task. For example, a training text and its label data are used<text,label> Indicates that, assuming that the label can be used for the first task (corresponding to L1) and the second task (corresponding to L2), the text can be input into Figure 7 In the text processing model shown, the result output from the first task processing model can be called the first prediction result, and the result output from the second task processing model can be called the second prediction result. L1 can be constructed based on the first prediction result and the label, and L2 can be constructed based on the second prediction result and the label. Since the label is not used for the third and fourth tasks, L3=L4=0, and L at this time is L1+L2. Afterwards, the back propagation (BP) algorithm can be used to adjust the model parameters. Among them, the model involved in parameter adjustment can include a pre-trained language model and a task processing model corresponding to the label data. For example, if the above-mentioned label data is used for the first and second tasks, the first task processing model and the second task processing model can be updated based on the loss function L at this time.

[0091] In this embodiment, the representation features of the training text are extracted, and multiple tasks are executed in parallel based on the representation features. Since the representation features are extracted and processed based on the representation features, pure linguistic design is not required, and problems caused by pure linguistic design are avoided. In addition, by executing multiple tasks in parallel, compared with the serial mode, the mutual dependence between tasks can be avoided, and the accuracy, efficiency, etc. are improved. Therefore, the training effect of the text processing model can be improved.

[0092] In addition, during training, multiple task processing models are jointly trained. For example, the loss function is the sum of the loss functions corresponding to each task processing model involved, which can more effectively use the training data of each task processing model. Each task processing model is no longer dependent on each other, the training data is not coupled, and the training data does not need to be extracted sequentially, and the training is easier to maintain. By executing each task processing model in parallel, the training speed can be accelerated.

[0093] Figure 8 FIG. 8 is a schematic diagram according to an eighth embodiment of the present disclosure. The present embodiment provides a text processing apparatus. As shown in the figure, the text processing apparatus 800 includes a regularization module 801, a feature extraction module 802, and a task execution module 803. Figure 8

[0094] The regularization module 801 is configured to perform regularization processing on the text to obtain a regularized text. The feature extraction module 802 is configured to extract representation features of the regularized text. The task execution module 803 is configured to execute multiple tasks in parallel based on the representation features to obtain a task result of each task in the multiple tasks.

[0095] In this embodiment, the representation features of the regularized text are extracted, and multiple tasks are executed in parallel based on the representation features. Since the representation features are extracted and processed based on the representation features, pure linguistic design is not required, and problems caused by pure linguistic design are avoided. In addition, by executing multiple tasks in parallel, compared with the serial mode, the mutual dependence between tasks can be avoided, and the accuracy, efficiency, etc. are improved. Therefore, the training effect of the text processing model can be improved.

[0096] In some embodiments, the multiple tasks include at least two of word segmentation, part-of-speech tagging, prosody prediction, multi-pronunciation word prediction, and tone change.

[0097] Through the above-mentioned word segmentation, part-of-speech tagging, etc. tasks, the text front-end module can be applied.

[0098] In some embodiments, the feature extraction module 802 is further configured to: use a pre-trained language model to extract features of the input regularized text to output the representation features of the regularized text.

[0099] ​By using the pre-trained language model to extract the representation features of the regularized text, the representation features with better expression capability can be obtained.

[0100] In some embodiments, the task execution module 803 is further configured to process the input representation features using the single deep learning model corresponding to each task to output the task results of the tasks.

[0101] By using the deep learning model for processing, the accuracy can be improved.

[0102] In some embodiments, the deep learning model is a classification network model.

[0103] The deep learning model uses an FC network model, which can simplify the implementation.

[0104] Figure 9 is a schematic diagram according to the ninth embodiment of the present disclosure, and the present embodiment provides a text processing model training apparatus. As shown in Figure 9 The text processing model training apparatus 900 includes a feature extraction module 901, a task execution module 902, a construction module 903, and an adjustment module 904.

[0105] The text processing model includes a feature extraction model and a plurality of task processing models.

[0106] The feature extraction module 901 is configured to extract features of input training text using the feature extraction model to output representation features of the training text; the task execution module 902 is configured to process the representation features using each task processing model in the plurality of task processing models to output prediction results of each task; the construction module 903 is configured to construct a loss function based on the prediction results and label data of the training text; and the adjustment module 904 is configured to adjust model parameters of the feature extraction model and model parameters of at least one task processing model in the plurality of task processing models based on the loss function.

[0107] In the present embodiment, by extracting the representation features of the training text and performing a plurality of tasks in parallel based on the representation features, since the representation features are extracted and processed based on the representation features, pure linguistic design can be avoided, and problems caused by pure linguistic design can be avoided. In addition, by performing a plurality of tasks in parallel, compared with a serial mode, mutual dependence between tasks can be avoided, and accuracy, efficiency, and the like can be improved. Therefore, the present embodiment can improve the training effect of the text processing model.

[0108] Furthermore, multiple task processing models are trained jointly during training. For example, the loss function is the sum of the corresponding loss functions of each task processing model involved, enabling more efficient use of the training data of each task processing model. Task processing models are no longer interdependent, training data is decoupled, and training data does not need to be extracted sequentially, making training easier to maintain. By executing each task processing model in parallel, training can be accelerated.

[0109] In some embodiments, the plurality of task processing models are used to perform at least two of the following tasks:

[0110] At least two of the following: word segmentation, part-of-speech tagging, prosody prediction, polyphone prediction, and tone change.

[0111] Through the above-mentioned tasks such as word segmentation and part-of-speech tagging, it can be applied to the text front-end module.

[0112] In some embodiments, the feature extraction model is a pre-trained language model; and / or, each task processing model is a classification network model.

[0113] By using a pre-trained language model to extract representation features of training text, representation features with better expressiveness can be obtained.

[0114] It can be understood that in the embodiments of the present disclosure, the same or similar contents in different embodiments can be referenced to each other.

[0115] It can be understood that the terms “first”, “second”, etc. in the embodiments of the present disclosure are only used for distinction and do not indicate the degree of importance, time sequence, etc.

[0116] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0118] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0119] like Figure 10 As shown, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 10010 into a random access memory (RAM) 1003. Various programs and data required for the operation of the electronic device 1000 can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0120] Multiple components in the electronic device 1000 are connected to the I / O interface 1005, including an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the electronic device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0121] The computing unit 1001 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the text processing method. For example, in some embodiments, the text processing method or the training method of the text processing model can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the text processing method or the training method of the text processing model described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the text processing method or the text processing model training method in any other appropriate manner (for example, by means of firmware).

[0122] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0124] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0127] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0128] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0129] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A text processing method, comprising: Regularizing the text to obtain regularized text; The regularization process is used to convert non-standard words into spoken words; The regularized text is a plain text; Using a pre-trained language model, feature extraction is performed on the input regularized text to output representation features of the regularized text; Using multiple task processing models, based on the representation features, executing multiple tasks in parallel to obtain task results for each of the multiple tasks; The pre-trained language model and the task processing model are trained based on training text and its label data, and the model parameters of the pre-trained language model and the model parameters of the task processing model corresponding to the label data are adjusted based on a loss function, and the loss function is constructed based on the sub-loss functions corresponding to the respective tasks, wherein the sub-loss function corresponding to the task for which the label data is used is constructed based on the label data, and if the label data is used for multiple tasks, the multiple sub-loss functions corresponding to the multiple tasks are constructed based on the same label data, and the sub-loss function corresponding to the task for which the label data is not used is set to 0; The task results of the tasks are input into an acoustic model to output acoustic features through the acoustic model, and the acoustic features are input into a vocoder to output speech corresponding to the text through the vocoder.

2. The method according to claim 1, wherein The multiple tasks include: At least two of the following: word segmentation, part-of-speech tagging, prosody prediction, polyphone prediction, and tone change.

3. The method according to claim 1, wherein The task processing model is a classification network model.

4. A method for training a text processing model, the text processing model comprising: A feature extraction model and a multiple task processing model, the method comprising: The feature extraction model is used to extract features from an input training text to output representation features of the training text; the training text is a plain text obtained by converting non-standard words into spoken words; Using each of the multiple task processing models to process the representation features in parallel to output a prediction result for each task; Constructing a loss function based on the prediction result and the label data of the training text; Based on the loss function, adjusting model parameters of the feature extraction model and model parameters of the task processing models corresponding to the label data in the multiple task processing models; The loss function is constructed based on the sub-loss functions corresponding to the respective tasks, wherein the sub-loss function corresponding to the task for which the label data is used is constructed based on the label data, and if the label data is used for multiple tasks, multiple sub-loss functions corresponding to the multiple tasks are constructed based on the same label data, and the sub-loss function corresponding to the task for which the label data is not used is set to 0; The text processing model is used to obtain the task results of each task during text processing. The task results of each task are input into the acoustic model to output acoustic features through the acoustic model. The acoustic features are input into the vocoder to output the speech corresponding to the text through the vocoder.

5. The method according to claim 4, wherein The multiple task processing models are used to perform at least two of the following tasks: At least two of the following: word segmentation, part-of-speech tagging, prosody prediction, polyphone prediction, and tone change.

6. The method according to claim 4 or 5, wherein: The feature extraction model is a pre-trained language model; and / or, Each of the task processing models is a classification network model.

7. A text processing device comprising: A regularization module is used to perform regularization processing on the text to obtain regularized text; The regularization process is used to convert non-standard words into spoken words; the regularized text is a plain text; A feature extraction module, configured to extract features from the input regularized text using a pre-trained language model to output representation features of the regularized text; a task execution module, configured to execute the plurality of tasks in parallel based on the representation features using a plurality of task processing models to obtain a task result for each of the plurality of tasks; The pre-trained language model and the task processing model are trained based on training text and its label data, and the model parameters of the pre-trained language model and the model parameters of the task processing model corresponding to the label data are adjusted based on a loss function, and the loss function is constructed based on the sub-loss functions corresponding to the respective tasks, wherein the sub-loss function corresponding to the task for which the label data is used is constructed based on the label data, and if the label data is used for multiple tasks, the multiple sub-loss functions corresponding to the multiple tasks are constructed based on the same label data, and the sub-loss function corresponding to the task for which the label data is not used is set to 0; The task results of the tasks are input into an acoustic model to output acoustic features through the acoustic model, and the acoustic features are input into a vocoder to output speech corresponding to the text through the vocoder.

8. The device according to claim 7, wherein The multiple tasks include: At least two of the following: word segmentation, part-of-speech tagging, prosody prediction, polyphone prediction, and tone change.

9. The device according to claim 7, wherein The task processing model is a classification network model.

10. A training device for a text processing model, the text processing model comprising: A feature extraction model and a plurality of task processing models, the apparatus comprising: A feature extraction module is used to extract features from an input training text using the feature extraction model to output representation features of the training text; the training text is a plain text obtained by converting non-standard words into spoken words; a task execution module, configured to process the representation features in parallel using each of the plurality of task processing models to output a prediction result for each task; A construction module, configured to construct a loss function based on the prediction result and the label data of the training text; An adjustment module, configured to adjust model parameters of the feature extraction model and model parameters of the task processing models corresponding to the label data in the plurality of task processing models based on the loss function; The loss function is constructed based on the sub-loss functions corresponding to the respective tasks, wherein the sub-loss function corresponding to the task for which the label data is used is constructed based on the label data, and if the label data is used for multiple tasks, multiple sub-loss functions corresponding to the multiple tasks are constructed based on the same label data, and the sub-loss function corresponding to the task for which the label data is not used is set to 0; The text processing model is used to obtain the task results of each task during text processing. The task results of each task are input into the acoustic model to output acoustic features through the acoustic model. The acoustic features are input into the vocoder to output the speech corresponding to the text through the vocoder.

11. The device according to claim 10, wherein The multiple task processing models are used to perform at least two of the following tasks: At least two of the following: word segmentation, part-of-speech tagging, prosody prediction, polyphone prediction, and tone change.

12. The device according to claim 10 or 11, wherein The feature extraction model is a pre-trained language model; and / or, Each of the task processing models is a classification network model.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

15. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images