Text processing method, device, computer equipment and storage medium

By using the prosthetic annotation model to generate a large amount of labeling data, the problems of high manual annotation cost and poor model stability in the existing technology are solved, and efficient prosthetic prediction model training and labeling are achieved.

CN114091444BActive Publication Date: 2024-11-01SOUNDAI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111350341.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-15
Publication Date
2024-11-01
Estimated Expiration
2041-11-15

AI Technical Summary

Technical Problem

In the prior art, text-based speech synthesis requires manual labeling of large amounts of data to train the prosody prediction model, resulting in high training costs, poor model stability, and inconsistent labeling standards.

Method used

A pronunciation model including the first identification network and annotation network is adopted. The pronunciation labeling model is trained through a small amount of sample data, and a large amount of annotation data is generated to train the pronunciation prediction model, reducing the cost of manual annotation and improving the labeling efficiency and model stability.

Benefits of technology

The pronunciation prediction model training without manual annotation is realized, which improves the model's training data coverage and stability, reduces training costs, and improves the labeling efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114091444B_ABST
    Figure CN114091444B_ABST
Patent Text Reader

Abstract

The embodiments of the present application disclose a text processing method, device, computer equipment and storage medium, which belong to the field of computer technology. The method includes: obtaining a rhythm annotation model, the rhythm annotation model includes a first recognition network and an annotation network, the first recognition network has the function of identifying each word in any text; based on multiple first sample data, training the rhythm annotation model; calling the trained rhythm annotation model, respectively annotating multiple second texts to obtain multiple second sample data; based on multiple second sample data, training the rhythm prediction model. This method does not require manual annotation of a large amount of sample data, and can also train an accurate rhythm prediction model. While ensuring the accuracy of the rhythm prediction model, it reduces the cost of manual annotation and improves the annotation efficiency, thereby improving the training efficiency of the rhythm prediction model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a text processing method, apparatus, computer equipment and storage medium. Background Art

[0002] With the development of computer technology and artificial intelligence technology, text-based speech synthesis is becoming more and more common. In order to improve the playback effect of synthesized speech, it is necessary to accurately obtain the prosodic label of each word in the text.

[0003] In the related art, a prosody prediction model is called to process text and obtain the prosody label of each word in the text. However, in order to achieve accurate prediction, a large amount of text needs to be manually annotated and the prosody prediction model needs to be trained using a large amount of annotated text, resulting in a high training cost for the prosody prediction model. Summary of the invention

[0004] The embodiments of the present application provide a text processing method, apparatus, computer device and storage medium, which do not require manual labeling of sample data for training a prosody prediction model, while improving the coverage of training data, improving the stability of the prosody prediction model, and reducing the training cost of the prosody prediction model. The technical solution is as follows:

[0005] In one aspect, a text processing method is provided, the method comprising:

[0006] Acquire a prosody annotation model, the prosody annotation model comprising a first recognition network and an annotation network, the first recognition network having a function of recognizing each word in any text, the annotation network being used for training a prosody tag of at least one word in any text;

[0007] Training the prosody annotation model based on a plurality of first sample data, wherein the first sample data includes a first text and a prosody label of each word in the first text;

[0008] Calling the trained prosody annotation model to annotate the plurality of second texts respectively to obtain a plurality of second sample data, wherein the second sample data includes the second text and the prosody label of each word in the second text, and the number of the plurality of first sample data is less than the number of the plurality of second sample data;

[0009] A prosody prediction model is trained based on the plurality of second sample data.

[0010] In a possible implementation, the training of the prosody annotation model based on the plurality of first sample data includes:

[0011] Calling the first recognition network to recognize the first text in the first sample data, and obtaining a plurality of words in the first text and an association relationship between each two adjacent words;

[0012] Calling the annotation network to annotate multiple words in the first text based on the association relationship between each two adjacent words in the first text, and obtaining a predicted prosodic label of the first word in each two adjacent words in the first text, and a predicted prosodic label of the last word in the first text;

[0013] The prosody annotation model is trained based on the prosody label of each word in the first text and the predicted prosody label.

[0014] In another possible implementation, calling the trained prosody annotation model to annotate the plurality of second texts respectively to obtain a plurality of second sample data includes:

[0015] Calling the first recognition network to recognize the second text, and obtaining a plurality of words in the second text and an association relationship between each two adjacent words;

[0016] The annotation network is called to annotate multiple words in the second text based on the association relationship between each two adjacent words in the second text, so as to obtain second sample data corresponding to the second text.

[0017] In another possible implementation, after training the prosody prediction model based on the plurality of second sample data, the method further includes:

[0018] The trained prosody prediction model is called to process the third text to obtain a prosody label for each word in the third text.

[0019] In another possible implementation, the trained prosody prediction model includes a second recognition network and a prosody prediction network, wherein the second recognition network has the function of recognizing each word in any text and the association relationship between every two adjacent words, and the prosody prediction network has the function of predicting the prosody label of at least one word in any text;

[0020] The calling of the trained prosody prediction model to process the third text to obtain a prosody label for each word in the third text includes:

[0021] calling the second recognition network to process the third text to obtain a plurality of words in the third text and an association relationship between each two adjacent words;

[0022] The prosody prediction network is called to perform prosody prediction on multiple words in the third text based on the association relationship between each two adjacent words in the third text, so as to obtain the prosody label of the first word in each two adjacent words in the third text and the prosody label of the last word in the third text.

[0023] In another possible implementation, the second recognition network is at least one of a long short-term memory network and a bidirectional long short-term memory network.

[0024] In another possible implementation, after calling the trained prosody prediction model to process the third text to obtain the prosody label of each word in the third text, the method further includes:

[0025] Based on the third text and the prosodic tag of each word in the third text, speech synthesis is performed on the third text to obtain speech corresponding to the third text.

[0026] In another possible implementation, the prosody tag includes at least a first tag and a second tag, wherein the first tag indicates that there is no pause between the word and the next word of the word, and the second tag indicates that there is a pause between the word and the next word of the word, and the length of the pause between the word and the next word of the word.

[0027] In another possible implementation, the first recognition network is a transformer-based bidirectional encoder representation BERT network.

[0028] In another aspect, a text processing device is provided, the device comprising:

[0029] A labeling model acquisition module, used to acquire a prosody labeling model, wherein the prosody labeling model includes a first recognition network and a labeling network, wherein the first recognition network has a function of recognizing each word in any text, and the labeling network is used to train a prosody label of at least one word in any text;

[0030] A first training module, configured to train the prosody annotation model based on a plurality of first sample data, wherein the first sample data includes a first text and a prosody label of each word in the first text;

[0031] a labeling module, configured to call the trained prosody labeling model to label the plurality of second texts respectively to obtain a plurality of second sample data, wherein the second sample data includes the second text and the prosody label of each word in the second text, and the number of the plurality of first sample data is less than the number of the plurality of second sample data;

[0032] The second training module is used to train a rhythm prediction model based on the plurality of second sample data.

[0033] In a possible implementation, the first training module is used to:

[0034] Calling the first recognition network to recognize the first text in the first sample data, and obtaining a plurality of words in the first text and an association relationship between each two adjacent words;

[0035] Calling the annotation network to annotate multiple words in the first text based on the association relationship between each two adjacent words in the first text, and obtaining a predicted prosodic label of the first word in each two adjacent words in the first text, and a predicted prosodic label of the last word in the first text;

[0036] The prosody annotation model is trained based on the prosody label of each word in the first text and the predicted prosody label.

[0037] In another possible implementation, the marking module is used to:

[0038] Calling the first recognition network to recognize the second text, and obtaining a plurality of words in the second text and an association relationship between each two adjacent words;

[0039] The annotation network is called to annotate multiple words in the second text based on the association relationship between each two adjacent words in the second text, so as to obtain second sample data corresponding to the second text.

[0040] In another possible implementation, the device further includes:

[0041] The prosody prediction module is used to call the trained prosody prediction model to process the third text to obtain a prosody label for each word in the third text.

[0042] In another possible implementation, the trained prosody prediction model includes a second recognition network and a prosody prediction network, wherein the second recognition network has the function of recognizing each word in any text and the association relationship between every two adjacent words, and the prosody prediction network has the function of predicting the prosody label of at least one word in any text;

[0043] The rhythm prediction module is used to:

[0044] calling the second recognition network to process the third text to obtain a plurality of words in the third text and an association relationship between each two adjacent words;

[0045] The prosody prediction network is called to perform prosody prediction on multiple words in the third text based on the association relationship between each two adjacent words in the third text, so as to obtain the prosody label of the first word in each two adjacent words in the third text and the prosody label of the last word in the third text.

[0046] In another possible implementation, the second recognition network is at least one of a long short-term memory network and a bidirectional long short-term memory network.

[0047] In another possible implementation, the device further includes:

[0048] The speech synthesis module is used to perform speech synthesis on the third text based on the third text and the prosodic tag of each word in the third text to obtain the speech corresponding to the third text.

[0049] In another possible implementation, the prosody tag includes at least a first tag and a second tag, wherein the first tag indicates that there is no pause between the word and the next word of the word, and the second tag indicates that there is a pause between the word and the next word of the word, and the length of the pause between the word and the next word of the word.

[0050] In another possible implementation, the first recognition network is a transformer-based bidirectional encoder representation BERT network.

[0051] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed in the text processing method described in the above aspects.

[0052] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed in the text processing method described in the above aspects.

[0053] On the other hand, a computer program product or a computer program is provided, wherein the computer program product or the computer program comprises a computer program code, wherein the computer program code is stored in a computer-readable storage medium, and wherein the computer program code is loaded and executed by a processor to implement the operations performed in the text processing method as described in the above aspects.

[0054] In the embodiment of the present application, since the first recognition network in the rhythm annotation model has the function of recognizing each word in the text, when training the rhythm annotation model, it is not necessary to train the function of recognizing each word in the text of the rhythm annotation model, and an accurate rhythm annotation model can be trained by using less first sample data, and then the trained rhythm annotation model is called to annotate a large number of second texts, and a large number of second sample data are obtained to train the rhythm prediction model, thereby improving the coverage rate of the training data used to train the rhythm prediction model, and an accurate rhythm prediction model can be trained without manually annotating a large number of sample data, while ensuring the accuracy of the rhythm prediction model, reducing the cost of manual annotation. Moreover, compared with manual annotation, the rhythm annotation model is used to annotate the second text, thereby improving the annotation efficiency, thereby improving the training efficiency of the rhythm prediction model.

[0055] Furthermore, since prosody annotation is subjective when manually annotating text, when a large amount of second sample data is annotated by multiple annotators, the annotation standards will be inconsistent, resulting in poor stability of the trained prosody prediction model. In the present application, the prosody annotation model is used for annotation, and a unified annotation standard is adopted, which will not be affected by subjective influences. Therefore, training the prosody prediction model based on a large amount of second sample data annotated by the prosody annotation model is beneficial to improving the stability of the prosody prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0058] Figure 2 is a flowchart of a text processing method provided by an embodiment of the present application;

[0059] Figure 3 is a flowchart of another text processing method provided by an embodiment of the present application;

[0060] Figure 4 is a structural diagram of a rhythmic annotation model provided in an embodiment of the present application;

[0061] Figure 5 is a structural schematic diagram of a rhythm prediction model provided in an embodiment of the present application;

[0062] Figure 6is a schematic diagram of the structure of another rhythm prediction model provided in an embodiment of the present application;

[0063] Figure 7 is a structural schematic diagram of a text processing device provided in an embodiment of the present application;

[0064] Figure 8 is a structural schematic diagram of another text processing device provided in an embodiment of the present application;

[0065] Fig. 9 is a schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0066] Fig.10 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0068] It is understood that the terms "first", "second", etc. used in this application can be used in this article to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of this application, a first text can be referred to as a second text, and a second text can be referred to as a first text.

[0069] The terms "at least one", "a plurality of", "each", "any" and the like used in this application include one, two or more than two, a plurality of includes two or more than two, each refers to each of the corresponding plurality, and any refers to any one of the plurality. For example, a plurality of words includes three words, and each word refers to each of the three words, and any refers to any one of the three words, which can be the first, the second, or the third.

[0070] An embodiment of the present application provides a text processing method, the execution subject of the method is a computer device, the computer device first obtains a rhythm annotation model, trains the rhythm annotation model based on a small amount of first sample data, then calls the trained rhythm annotation model to annotate a large amount of second text to obtain a large amount of second sample data, trains a rhythm prediction model based on the large amount of second sample data, and then processes the text based on the trained rhythm prediction model.

[0071] Optionally, the computer device is a terminal, such as a mobile phone, a desktop computer, a laptop computer, a tablet computer, a smart TV, a smart speaker, a vehicle terminal, an intelligent robot, etc. Alternatively, the computer device is a server, which may be a single server, a server cluster consisting of several servers, or a cloud computing server center.

[0072] In one possible implementation, see Figure 1 In the implementation environment shown, the text processing method provided by the embodiment of the present application is interactively executed by the terminal 101 and the server 102. The server 102 obtains and trains a rhythm annotation model, calls the trained rhythm annotation model, annotates a large amount of second texts, obtains a large amount of second sample data, trains a rhythm prediction model based on the large amount of second sample data, sends the trained rhythm prediction model to the terminal 101, and the terminal 101 uses the rhythm prediction model to process the text.

[0073] The method provided in the embodiment of the present application can be applied to a variety of scenarios, such as voice mailboxes, telephone banks, station broadcast systems, elevator broadcast systems, elevator digital human systems, voice notification systems, hospital call systems, etc.

[0074] Taking the application in the elevator digital human system as an example, texts corresponding to different situations are set in advance in the elevator digital human system, and the rhythm prediction model trained by the embodiment of the present application is used to predict the rhythm label of each word in the text, and then the text is synthesized into the corresponding speech based on the predicted rhythm label, and then the synthesized speech is played to interact with users taking the elevator.

[0075] Figure 2 1 is a flowchart of a text processing method provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a computer device. Figure 2 , the method comprises the following steps:

[0076] 201. A computer device obtains a prosody annotation model, wherein the prosody annotation model includes a first recognition network and an annotation network, wherein the first recognition network has a function of recognizing each word in any text, and the annotation network is used to train a prosody tag of at least one word in any text.

[0077] The prosodic annotation model is used to process any text to annotate the prosodic label of at least one word in the text. The first recognition network is a trained network that has the function of identifying each word in the text. In the subsequent training process of the prosodic annotation model, the first recognition network is continuously trained so that the first recognition network has the function of identifying the association between each two adjacent words in the text. The annotation network is essentially a classification network that can classify each word based on the association between each two adjacent words identified by the first recognition network, and determine the prosodic label of each word based on the classification result. The prosodic label at least indicates whether there is a pause between a word and the next word of the word.

[0078] 202. The computer device trains a prosody labeling model based on the plurality of first sample data.

[0079] The first sample data includes the first text and the prosodic label of each word in the first text, and the first sample data is obtained by manually annotating the first text. The first text is any sentence, and the embodiment of the present application does not limit the content of the first text.

[0080] Since the first recognition network in the prosody annotation model has the function of recognizing each word in the text, there is no need to train the function of recognizing each word in the text during the training of the prosody annotation model, so that an accurate prosody annotation model can be trained with less sample data.

[0081] 203. The computer device calls the trained prosody annotation model to annotate the plurality of second texts respectively to obtain a plurality of second sample data, wherein the number of the plurality of first sample data is less than the number of the plurality of second sample data.

[0082] The second sample data includes the second text and the prosodic label of each word in the second text. The second text is any sentence, and the embodiment of the present application does not limit the content of the second text.

[0083] The computer device calls the trained prosody annotation model to annotate the second text, and outputs each word in the second text and the prosody label of each word. Since the networks in the prosody prediction model that needs to be trained later are all untrained, a large amount of sample data is required to train an accurate prosody prediction model. Therefore, the prosody annotation model is called to annotate a large amount of second sample data, which is convenient for the subsequent training of the prosody prediction model, and there is no need to manually annotate the sample data, thereby reducing the training cost of the prosody prediction model.

[0084] 204. The computer device trains a rhythm prediction model based on the plurality of second sample data.

[0085] The prosody prediction model is used to process any text to predict the prosody label of at least one word in the text. The computer device iteratively trains the prosody prediction model based on multiple second sample data to obtain a trained prosody prediction model. Afterwards, the computer device can call the trained prosody prediction model to process the text.

[0086] In the method provided by the embodiment of the present application, since the first recognition network in the rhythm annotation model has the function of recognizing each word in the text, when training the rhythm annotation model, it is not necessary to further train the function of recognizing each word in the text of the rhythm annotation model, and an accurate rhythm annotation model can be trained by using less first sample data, and then the trained rhythm annotation model is called to annotate a large number of second texts, so as to obtain a large number of second sample data to train the rhythm prediction model, thereby improving the coverage rate of the training data used to train the rhythm prediction model, thereby improving the stability of the rhythm prediction model, and an accurate rhythm prediction model can be trained without manually annotating a large number of sample data, thereby reducing the cost of manual annotation while ensuring the accuracy of the rhythm prediction model. In addition, the rhythm annotation model is used to annotate the second text, thereby improving the annotation efficiency, thereby improving the training efficiency of the rhythm prediction model.

[0087] Above Figure 2 The embodiment shown briefly introduces the process of training the rhythm prediction model. Figure 3 The illustrated embodiment introduces in detail the process of training a prosody prediction model, the process of using the prosody prediction model, and the speech synthesis process.

[0088] Figure 3 1 is a flowchart of a text processing method provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a computer device. Figure 3 , the method comprises the following steps:

[0089] 301. The computer device obtains a rhythmic annotation model.

[0090] The prosodic tagging model is used to process any text to tag the prosodic tag of at least one word in the text. Figure 4 The model structure of the prosody annotation model shown in FIG. 4 includes a first recognition network 401 and an annotation network 402. The first recognition network 401 is a trained network that has the function of identifying each word in the text. In the subsequent training process of the prosody annotation model, the first recognition network is continuously trained so that the first recognition network has the function of identifying the association between each two adjacent words in the text.

[0091] Optionally, the first recognition network is a BERT (Bidirectional Encoder Representations from Transformers) network, which contains pre-trained information corresponding to words. Since the BERT network uses a multi-layer Transformer to process text bidirectionally, it can accurately learn the contextual relationship between multiple words in the text, and since the BERT network has the function of identifying each word in the text, there is no need to use a large amount of sample data to train the BERT network in the subsequent training process. Alternatively, the first recognition network is other networks that have the function of identifying each word in the text.

[0092] The annotation network 402 in the prosody annotation model is essentially a classification network, which can classify each word based on the association between each two adjacent words identified by the first recognition network, and determine the prosody label of each word based on the classification result. Optionally, the annotation network is a CRF (Conditional Random Field) network or other networks.

[0093] The prosody tag at least indicates whether there is a pause between a word and the next word of the word. Optionally, the prosody tag includes at least a first tag and a second tag, the first tag indicating that there is no pause between the word and the next word of the word, the second tag indicating that there is a pause between the word and the next word of the word, and the pause duration between the word and the next word of the word. The pause duration is any duration, for example, the pause duration is 0.1 seconds, 0.2 seconds or other durations. For example, the first tag is nobreak and the second tag is break.

[0094] In a possible implementation, the second tag includes a plurality of subtags, each of which indicates that there is a pause between a word and the next word of the word, and the difference is that the pause duration corresponding to each subtag is different.

[0095] 302. The computer device obtains a plurality of first sample data.

[0096] The plurality of first sample data are manually annotated sample data. The first sample data include a first text and a prosodic label of each word in the first text.

[0097] In one possible implementation, a computer device first obtains a first text, which is a Chinese text, an English text, or a text of another language type, which is not limited in the embodiments of the present application. The first text is any sentence, and the embodiments of the present application do not limit the content of the first text. For example, the first text is "I have completed my work today", or the first text is "Please turn right 100 meters ahead", etc. Then the annotator annotates the multiple first texts respectively to obtain multiple first sample data. That is, for any first text, the annotator divides the first text into multiple words according to the understanding of the first text, and then annotates each word with a rhythmic label, thereby obtaining the first sample data corresponding to the first text.

[0098] The above implementation is only described by taking the acquisition of one first sample data as an example. For a plurality of first texts, the above implementation can be used to acquire the corresponding first sample data.

[0099] Optionally, since there is a certain degree of subjectivity when annotators annotate prosodic tags for each word in the text, the prosodic tags annotated by different annotators for the same first text may be different. Therefore, in order to ensure the stability of prosodic tagging, the same annotator annotates multiple first texts.

[0100] It should be noted that the embodiment of the present application is only described by taking the example of first obtaining the prosody annotation model and then obtaining the first sample data. In another embodiment, the first sample data can be obtained first, and then the prosody annotation model can be obtained, or the first sample data and the prosody annotation model can be obtained at the same time. The embodiment of the present application does not limit the order of execution of the steps.

[0101] 303. The computer device trains a prosody labeling model based on the plurality of first sample data.

[0102] The computer device performs multiple iterative training on the prosody annotation model based on multiple first sample data to obtain a trained prosody annotation model. Optionally, the prosody annotation model can be iteratively trained once or multiple times based on one first sample data. The training process of the prosody annotation model is described below by taking one iterative training process as an example.

[0103] In a possible implementation, a computer device calls a first recognition network to recognize a first text in a first sample data, and obtains a plurality of words in the first text and an association relationship between each two adjacent words; then calls an annotation network to annotate a plurality of words in the first text based on the association relationship between each two adjacent words in the first text, and obtains a predicted prosodic label of the first word in each two adjacent words in the first text, and a predicted prosodic label of the last word in the first text; and trains a prosodic labeling model based on the prosodic label and the predicted prosodic label of each word in the first text, that is, the computer device adjusts the model parameters of the prosodic labeling model based on the difference between the prosodic label and the predicted prosodic label of each word in the first text, so as to reduce the difference between the prosodic label and the predicted prosodic label of each word in the first text. The predicted prosodic label of the last word in the first text defaults to a pause.

[0104] Optionally, the computer device calls the first recognition network, extracts features from the input first text, obtains text features of the first text, processes the text features, obtains semantic features of the first text, divides the first text based on the semantic features, obtains multiple words in the first text, and determines the association relationship between each two adjacent words based on the contextual relationship of the multiple words in the first text represented by the semantic features. The association relationship between two adjacent words at least indicates whether there is a pause between the two words.

[0105] Optionally, the first recognition network outputs a plurality of words and prosodic features in the first text, the prosodic features indicating the association between each two adjacent words, for example, the prosodic features are in a matrix, vector or other form. The computer device calls the annotation network to process the input plurality of words and prosodic features to obtain a prosodic label for each word.

[0106] In a possible implementation, a labeling network is called to process multiple words and prosodic features to obtain the prediction probability corresponding to each word. When the prediction probability corresponding to a word is not greater than the reference probability, the prosodic label of the word is determined to be the first label, that is, there is no pause between the word and the next word. When the prediction probability corresponding to the word is greater than the reference probability, the prosodic label of the word is determined to be the second label, that is, there is a pause between the word and the next word. Optionally, when there is a pause between the word and the next word, the pause duration between the word and the next word is determined according to the size of the prediction probability. The greater the prediction probability, the longer the pause duration, and the smaller the prediction probability, the shorter the pause duration.

[0107] The reference probability is any value greater than 0 and less than 1, for example, the reference probability is 0.5, 0.6 or other values.

[0108] 304. The computer device calls the trained prosody annotation model to annotate the plurality of second texts respectively to obtain a plurality of second sample data.

[0109] In an embodiment of the present application, after a small amount of first sample data is used to train a prosody annotation model, the prosody annotation model can be called to annotate multiple second texts to obtain multiple second sample data for training a prosody prediction model. The multiple second sample data do not require manual annotation, thereby reducing the annotation cost.

[0110] The second text is a Chinese text, an English text, or a text of another language type, which is not limited in the present embodiment. The second text is any sentence, which is not limited in the present embodiment. Optionally, the second text is the same as the first text, or the second text is different from the first text.

[0111] In one possible implementation, in order to improve the accuracy of the prosody prediction model trained subsequently, it is necessary to increase the word coverage, that is, it is necessary to obtain a large amount of text in multiple fields and expand the data source of the text. For example, the second text is obtained from some articles, forums or news on the Internet. The embodiment of the present application does not limit the way to obtain the second text.

[0112] For any second text, in one possible implementation, the computer device calls the first recognition network to recognize the second text to obtain multiple words in the second text and the association relationship between each two adjacent words; calls the annotation network to annotate the multiple words in the second text based on the association relationship between each two adjacent words in the second text to obtain second sample data corresponding to the second text, and the second text data includes the second text and the prosodic label of each word in the second text.

[0113] The above annotation process is only described by taking the annotation of one second text as an example. Multiple second texts can be annotated using the above annotation process to obtain multiple second sample data.

[0114] The implementation method of the computer device calling the prosody annotation model to obtain the second sample data is the same as the process of the prosody annotation model processing the first text, which will not be repeated here.

[0115] 305. The computer device trains a rhythm prediction model based on the plurality of second sample data.

[0116] The prosody prediction model is used to process any text to predict the prosody label of at least one word in the text. Figure 5The model structure of the prosody prediction model shown in FIG. 5 includes a second recognition network 501 and a prosody prediction network 502. The training goal of the second recognition network 501 is to have the function of recognizing each word in any text, and to have the function of recognizing the association between each two adjacent words in the text, which is the same as the function of the first recognition network. The training goal of the prosody prediction network 502 is to have the function of predicting the prosody label of each word in any text.

[0117] Optionally, the second recognition network is a LSTM (Long Short Term Memory) network, a Bi-LSTM (Bidirectional Long Short Term Memory) network, a HMM (HiddenMarkov Model) or other networks. Optionally, the rhythm prediction network is a CRF network or other networks.

[0118] The computer device performs multiple iterative training on the prosody prediction model based on the multiple second sample data to obtain a trained prosody prediction model. The multiple second sample data are completely different from the multiple first sample data, or the multiple second sample data include some sample data in the multiple first sample data, or the multiple second sample data include all the first sample data. Optionally, the prosody prediction model can be iteratively trained once or multiple times based on one second sample data. The training process of the prosody prediction model is described below by taking one iterative training process as an example.

[0119] In one possible implementation, the computer device calls a second recognition network to recognize the second text in the second sample data to obtain multiple words in the second text and the association relationship between each two adjacent words; then calls a prediction network to predict the rhythmic label of each word in the second text based on the association relationship between each two adjacent words in the second text; and trains a rhythmic prediction model based on the rhythmic label of each word marked in the second sample data and the predicted rhythmic label of each word, that is, the computer device adjusts the model parameters of the rhythmic prediction model based on the difference between the rhythmic label of each word marked in the second sample data and the predicted rhythmic label of each word.

[0120] Optionally, the computer device calls the second recognition network, extracts features from the input second text, obtains text features of the second text, processes the text features, obtains semantic features of the second text, divides the second text based on the semantic features, obtains multiple words in the second text, and determines the association relationship between each two adjacent words based on the contextual relationship of the multiple words in the second text represented by the semantic features. The association relationship between two adjacent words at least indicates whether there is a pause between the two words.

[0121] Optionally, the second recognition network outputs a plurality of words and prosodic features in the second text, the prosodic features indicating the association between each two adjacent words, for example, the prosodic features are in a matrix, vector or other form. The computer device calls the annotation network to process the input plurality of words and prosodic features to obtain a prosodic label for each word.

[0122] In a possible implementation, a prosody prediction network is called to process multiple words and prosody features to obtain the prediction probability corresponding to each word. When the prediction probability corresponding to the word is not greater than the reference probability, the prosody label of the word is determined to be the first label, that is, there is no pause between the word and the next word. When the prediction probability corresponding to the word is greater than the reference probability, the prosody label of the word is determined to be the second label, that is, there is a pause between the word and the next word. Optionally, when there is a pause between the word and the next word, the pause duration between the word and the next word is determined according to the size of the prediction probability. The larger the prediction probability, the longer the pause duration, and the smaller the prediction probability, the shorter the pause duration. The reference probability is any value greater than 0 and less than 1, for example, the reference probability is 0.5, 0.6 or other values.

[0123] In the above implementation, although the functions of the second recognition network and the first recognition network are the same, since the first recognition network is a pre-trained network, for example, the first recognition network is a BERT network, and a pre-trained network not only has one function, but also has commonly used text processing functions such as classification function and sequence labeling function, since the first recognition network needs to implement more functions, the network structure of the first recognition network is relatively complex; and the second recognition network is an untrained network, for example, the second recognition network is an LSTM network and a Bi-LSTM network. In the rhythmic labeling scenario, the second recognition network is trained to recognize multiple words in the text and the association relationship between each two adjacent words. The second recognition network does not need to implement additional functions. Therefore, compared with the network structure of the first recognition network, the network structure of the second recognition network is simpler. Although both the prosody annotation network and the prosody prediction network can predict the prosody label of each word in the text, the first recognition network in the prosody annotation model still needs to implement some additional text processing functions, while the network structure of the second recognition network in the prosody prediction model is relatively simple and does not require text classification and annotation. Therefore, from the perspective of model operation efficiency, the prosody prediction model has higher operation efficiency and is more suitable for practical applications than the prosody annotation model.

[0124] 306. The computer device calls the trained prosody prediction model to process the third text to obtain a prosody label for each word in the third text.

[0125] The third text is any text to be processed, for example, "We have arrived at the 7th floor", "Please go to window No. 1 to queue up", etc.

[0126] In one possible implementation, the computer device calls the second recognition network to process the third text to obtain multiple words in the third text and the association relationship between each two adjacent words; calls the rhythm prediction network to perform rhythm prediction on the multiple words in the third text based on the association relationship between each two adjacent words in the third text to obtain the rhythm label of the first word in each two adjacent words in the third text, and the rhythm label of the last word in the third text.

[0127] The implementation method of the computer device calling the prosody prediction model to obtain the prosody label of each word in the third text is the same as the processing process of the prosody prediction model on the second text, which will not be repeated here.

[0128] Taking the second recognition network as a Bi-LSTM network as an example, the model structure of the rhythm prediction model is as follows Figure 6As shown, the third text is "rhythm structure prediction framework". The third text is input into the rhythm prediction model. The third text is first processed by the embedding layer in the second recognition network to determine the word features x of multiple words "rhythm", "structure", "prediction" and "framework" in the third text. For multiple word features, they are processed by the forward long short-term memory layer (Forward-LSTM) and the backward long short-term memory layer (Backward-LSTM) respectively to obtain the forward feature m and the backward feature n. The forward feature m and the backward feature n are fused to obtain the context feature y of each word. The context feature y of each word is input into the rhythm prediction network to obtain the rhythm label of each word.

[0129] 307. The computer device performs speech synthesis on the third text based on the third text and the prosodic tag of each word in the third text to obtain speech corresponding to the third text.

[0130] TTS (Text To Speech) technology can convert text into speech. In the embodiment of the present application, after the computer device obtains the prosodic label of each word in the third text, it can perform speech synthesis on the third text to obtain the speech corresponding to the third text, that is, convert the third text into speech and play it out. When playing the speech, each word needs to pause according to the corresponding prosodic label.

[0131] In one possible implementation, the computer device searches for the speech corresponding to each word from a speech library, splices the speech corresponding to multiple words together, and obtains the speech corresponding to the third text. When splicing, whether there is a pause between two adjacent words and the length of the pause when there is a pause are determined based on the prosodic label; or, the computer device calls a speech synthesis model to process each word in the third text to obtain the acoustic features of each word, and then converts the third text into the corresponding speech based on the acoustic features and prosodic labels of each word; or the computer device can also synthesize speech in other ways. In the embodiment of the present application, the method of synthesizing the speech corresponding to the third text based on the third text and the prosodic label of each word in the third text is not limited.

[0132] It should be noted that the above embodiment is only described by taking the prosody tag indicating whether there is a pause between two adjacent words as an example. In another embodiment, the prosody tag also indicates the pitch and intensity of the words. Among them, the pitch refers to the high or low pitch of the sound emitted, and the intensity refers to the strength of the sound emitted.

[0133] Another point that needs to be explained is that the above embodiment is only described by taking the execution of steps 301-307 by a computer device as an example. In another embodiment, the above steps 301-3-7 can be executed interactively by the terminal and the server. For example, steps 301-306 are executed by the server and step 307 is executed by the terminal; or 301-305 are executed by the server and steps 306-307 are executed by the terminal; or other interactive methods can be used, and the embodiments of the present application are not limited to this.

[0134] In the method provided by the embodiment of the present application, since the first recognition network in the rhythm annotation model has the function of recognizing each word in the text, when training the rhythm annotation model, it is not necessary to further train the function of recognizing each word in the text of the rhythm annotation model, and an accurate rhythm annotation model can be trained by using less first sample data, and then the trained rhythm annotation model is called to annotate a large number of second texts, so as to obtain a large number of second sample data to train the rhythm prediction model, thereby improving the coverage rate of the training data used to train the rhythm prediction model, and an accurate rhythm prediction model can be trained without manually annotating a large number of sample data, thereby ensuring the accuracy of the rhythm prediction model and reducing the cost of manual annotation. Moreover, compared with manual annotation, the rhythm annotation model is used to annotate the second text, thereby improving the annotation efficiency, thereby improving the training efficiency of the rhythm prediction model.

[0135] Furthermore, since prosody annotation is subjective when manually annotating text, when a large amount of second sample data is annotated by multiple annotators, the annotation standards will be inconsistent, resulting in poor stability of the trained prosody prediction model. In the present application, the prosody annotation model is used for annotation, and a unified annotation standard is adopted, which will not be affected by subjective influences. Therefore, training the prosody prediction model based on a large amount of second sample data annotated by the prosody annotation model is beneficial to improving the stability of the prosody prediction model.

[0136] Moreover, since the network structure of the rhythm prediction model is simpler, it can process text faster than the rhythm annotation model. Therefore, in actual application scenarios, the use of the rhythm prediction model can achieve faster text processing and improve the text processing speed.

[0137] Figure 7 is a structural diagram of a text processing device provided in an embodiment of the present application. Figure 7 , the device comprises:

[0138] The annotation model acquisition module 701 is used to acquire a prosodic annotation model, wherein the prosodic annotation model includes a first recognition network and an annotation network, wherein the first recognition network has a function of recognizing each word in any text, and the annotation network is used to train and annotate a prosodic label of at least one word in any text;

[0139] A first training module 702, configured to train a prosody annotation model based on a plurality of first sample data, wherein the first sample data includes a first text and a prosody label of each word in the first text;

[0140] The annotation module 703 is used to call the trained prosody annotation model to respectively annotate the plurality of second texts to obtain a plurality of second sample data, wherein the second sample data includes the second texts and the prosody label of each word in the second texts, and the number of the plurality of first sample data is less than the number of the plurality of second sample data;

[0141] The second training module 704 is used to train a rhythm prediction model based on a plurality of second sample data.

[0142] In the device provided by the embodiment of the present application, since the first recognition network in the rhythm annotation model has the function of recognizing each word in the text, when training the rhythm annotation model, it is not necessary to further train the function of recognizing each word in the text of the rhythm annotation model, and an accurate rhythm annotation model can be trained by using less first sample data, and then the trained rhythm annotation model is called to annotate a large number of second texts, so as to obtain a large number of second sample data to train the rhythm prediction model, thereby improving the coverage rate of the training data used to train the rhythm prediction model, and an accurate rhythm prediction model can be trained without manually annotating a large number of sample data, thereby ensuring the accuracy of the rhythm prediction model and reducing the cost of manual annotation. Moreover, compared with manual annotation, the rhythm annotation model is used to annotate the second text, thereby improving the annotation efficiency, thereby improving the training efficiency of the rhythm prediction model.

[0143] In a possible implementation, the first training module 702 is used to:

[0144] Calling a first recognition network to recognize a first text in the first sample data, and obtaining a plurality of words in the first text and an association relationship between each two adjacent words;

[0145] Calling the annotation network to annotate multiple words in the first text based on the association relationship between each two adjacent words in the first text, and obtaining a predicted prosodic label of the first word in each two adjacent words in the first text, and a predicted prosodic label of the last word in the first text;

[0146] A prosodic labeling model is trained based on the prosodic label of each word in the first text and the predicted prosodic label.

[0147] In another possible implementation, the marking module 703 is used to:

[0148] Calling the first recognition network to recognize the second text, and obtaining a plurality of words in the second text and an association relationship between each two adjacent words;

[0149] The annotation network is called to annotate multiple words in the second text based on the association relationship between each two adjacent words in the second text to obtain second sample data corresponding to the second text.

[0150] In another possible implementation, see Figure 8 , the device further comprises:

[0151] The prosody prediction module 705 is used to call the trained prosody prediction model to process the third text to obtain a prosody label for each word in the third text.

[0152] In another possible implementation, the trained prosody prediction model includes a second recognition network and a prosody prediction network, wherein the second recognition network has the function of recognizing each word in any text and the association between each two adjacent words, and the prosody prediction network has the function of predicting the prosody label of at least one word in any text; see Figure 8 , the rhythm prediction module 705 is used to:

[0153] Calling the second recognition network to process the third text to obtain a plurality of words in the third text and an association relationship between each two adjacent words;

[0154] The prosody prediction network is called to perform prosody prediction on multiple words in the third text based on the association relationship between each two adjacent words in the third text, and the prosody label of the first word in each two adjacent words in the third text and the prosody label of the last word in the third text are obtained.

[0155] In another possible implementation, the second recognition network is at least one of a long short-term memory network and a bidirectional long short-term memory network.

[0156] In another possible implementation, see Figure 8 , the device further comprises:

[0157] The speech synthesis module 706 is used to perform speech synthesis on the third text based on the third text and the prosodic tag of each word in the third text to obtain speech corresponding to the third text.

[0158] In another possible implementation, the prosody tag includes at least a first tag and a second tag, the first tag indicating that there is no pause between a word and the next word, the second tag indicating that there is a pause between the word and the next word, and the length of the pause between the word and the next word.

[0159] In another possible implementation, the first recognition network is a transformer-based bidirectional encoder characterization BERT network.

[0160] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0161] It should be noted that: the text processing device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate when processing text. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the text processing device provided in the above embodiment and the text processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0162] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the operations performed in the text processing method of the above embodiment.

[0163] Optionally, the computer device is provided as a terminal. Fig. 9 1 is a schematic diagram of the structure of a terminal 900 provided in an embodiment of the present application. The terminal 900 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 900 may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0164] The terminal 900 includes a processor 901 and a memory 902 .

[0165] The processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0166] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one computer program, which is used to be executed by the processor 901 to implement the text processing method provided in the method embodiment of the present application.

[0167] In some embodiments, the terminal 900 may further optionally include: a peripheral device interface 903 and at least one peripheral device. The processor 901, the memory 902 and the peripheral device interface 903 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 903 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907, a positioning assembly 908 and a power supply 909.

[0168] The peripheral device interface 903 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0169] The radio frequency circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 904 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 904 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 904 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0170] The display screen 905 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 also has the ability to collect touch signals on the surface or above the surface of the display screen 905. The touch signal can be input to the processor 901 as a control signal for processing. At this time, the display screen 905 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 905 can be one, set on the front panel of the terminal 900; in other embodiments, the display screen 905 can be at least two, respectively set on different surfaces of the terminal 900 or in a folding design; in other embodiments, the display screen 905 can be a flexible display screen, set on the curved surface or folding surface of the terminal 900. Even, the display screen 905 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 905 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0171] The camera assembly 906 is used to capture images or videos. Optionally, the camera assembly 906 includes a front camera and a rear camera. The front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0172] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 901 for processing, or input them into the radio frequency circuit 904 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 900. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 907 may also include a headphone jack.

[0173] The positioning component 908 is used to locate the current geographical location of the terminal 900 to implement navigation or LBS (Location Based Service). The positioning component 908 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas positioning system, or the European Union's Galileo positioning system.

[0174] The power supply 909 is used to power various components in the terminal 900. The power supply 909 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 909 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0175] In some embodiments, the terminal 900 further includes one or more sensors 910 , including but not limited to: an acceleration sensor 911 , a gyroscope sensor 912 , a pressure sensor 913 , a fingerprint sensor 914 , an optical sensor 915 , and a proximity sensor 916 .

[0176] The acceleration sensor 911 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 900. For example, the acceleration sensor 911 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 901 can control the display screen 905 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 911. The acceleration sensor 911 can also be used to collect motion data of games or users.

[0177] The gyro sensor 912 can detect the body direction and rotation angle of the terminal 900, and the gyro sensor 912 can cooperate with the acceleration sensor 911 to collect the user's 3D actions on the terminal 900. The processor 901 can implement the following functions based on the data collected by the gyro sensor 912: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0178] The pressure sensor 913 can be set on the side frame of the terminal 900 and / or the lower layer of the display screen 905. When the pressure sensor 913 is set on the side frame of the terminal 900, it can detect the user's holding signal of the terminal 900, and the processor 901 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 913. When the pressure sensor 913 is set on the lower layer of the display screen 905, the processor 901 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 905. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0179] The fingerprint sensor 914 is used to collect the user's fingerprint, and the processor 901 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 914, or the fingerprint sensor 914 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 901 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 914 can be set on the front, back, or side of the terminal 900. When a physical button or a manufacturer logo is set on the terminal 900, the fingerprint sensor 914 can be integrated with the physical button or the manufacturer logo.

[0180] The optical sensor 915 is used to collect the ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the display screen 905 according to the ambient light intensity collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display brightness of the display screen 905 is increased; when the ambient light intensity is low, the display brightness of the display screen 905 is reduced. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera component 906 according to the ambient light intensity collected by the optical sensor 915.

[0181] The proximity sensor 916, also called a distance sensor, is disposed on the front panel of the terminal 900. The proximity sensor 916 is used to collect the distance between the user and the front of the terminal 900. In one embodiment, when the proximity sensor 916 detects that the distance between the user and the front of the terminal 900 is gradually decreasing, the processor 901 controls the display screen 905 to switch from the screen-on state to the screen-off state; when the proximity sensor 916 detects that the distance between the user and the front of the terminal 900 is gradually increasing, the processor 901 controls the display screen 905 to switch from the screen-off state to the screen-on state.

[0182] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation on the terminal 900, and the terminal 900 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0183] Optionally, the computer device is provided as a server. Fig.10 This is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 1000 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1001 and one or more memories 1002, wherein the memory 1002 stores at least one computer program, and the at least one computer program is loaded and executed by the processor 1001 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the server may also include other components for implementing device functions, which will not be described in detail here.

[0184] An embodiment of the present application also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed in the text processing method of the above embodiment.

[0185] The present application embodiment also provides a computer program product or computer program, which includes a computer program code, and the computer program code is stored in a computer-readable storage medium. The processor of the computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device implements the operations performed in the text processing method of the above embodiment.

[0186] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0187] The above are only optional embodiments of the embodiments of the present application and are not intended to limit the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A text processing method, It is characterized in that The method comprises: Acquire a prosody annotation model, wherein the prosody annotation model includes a first recognition network and an annotation network, wherein the first recognition network is a pre-trained network, the first recognition network has a function of recognizing each word in any text, and the annotation network is used to train and annotate a prosody label of at least one word in any text; Based on a plurality of first sample data, the prosody annotation model is trained so that the first recognition network has a function of identifying the association relationship between each two adjacent words in the text by processing the input text, and the annotation network can determine the prosody label of each word based on the association relationship between each two adjacent words identified by the first recognition network, wherein the first sample data includes a first text and the prosody label of each word in the first text, and the association relationship between two adjacent words at least indicates whether there is a pause between the two words; Calling the trained prosody annotation model to annotate the plurality of second texts respectively to obtain a plurality of second sample data, wherein the second sample data includes the second text and the prosody label of each word in the second text, and the number of the plurality of first sample data is less than the number of the plurality of second sample data; Based on the multiple second sample data, a rhythm prediction model is trained. The trained rhythm prediction model includes a second recognition network and a rhythm prediction network. The second recognition network has the function of identifying each word in any text and the association between every two adjacent words. The rhythm prediction network has the function of predicting the rhythm label of at least one word in any text according to the recognition result of the second recognition network.

2. The method according to claim 1, It is characterized in that The step of training the prosody annotation model based on the plurality of first sample data includes: Calling the first recognition network to recognize the first text in the first sample data, and obtaining a plurality of words in the first text and an association relationship between each two adjacent words; Calling the annotation network to annotate multiple words in the first text based on the association relationship between each two adjacent words in the first text, and obtaining a predicted prosodic label of the first word in each two adjacent words in the first text, and a predicted prosodic label of the last word in the first text; The prosody annotation model is trained based on the prosody label of each word in the first text and the predicted prosody label.

3. The method according to claim 1, It is characterized in that The calling of the trained prosody annotation model to respectively annotate the plurality of second texts to obtain a plurality of second sample data includes: Calling the first recognition network to recognize the second text, and obtaining a plurality of words in the second text and an association relationship between each two adjacent words; The annotation network is called to annotate multiple words in the second text based on the association relationship between each two adjacent words in the second text, so as to obtain second sample data corresponding to the second text.

4. The method according to claim 1, It is characterized in that After training the rhythm prediction model based on the plurality of second sample data, the method further includes: The trained prosody prediction model is called to process the third text to obtain a prosody label for each word in the third text.

5. The method according to claim 4, It is characterized in that The calling of the trained prosody prediction model to process the third text to obtain a prosody label for each word in the third text includes: calling the second recognition network to process the third text to obtain a plurality of words in the third text and an association relationship between each two adjacent words; The prosody prediction network is called to perform prosody prediction on multiple words in the third text based on the association relationship between each two adjacent words in the third text, so as to obtain the prosody label of the first word in each two adjacent words in the third text and the prosody label of the last word in the third text.

6. The method according to claim 5, It is characterized in that The second recognition network is at least any one of a long short-term memory network and a bidirectional long short-term memory network.

7. The method according to claim 4, It is characterized in that After calling the trained prosody prediction model to process the third text to obtain the prosody label of each word in the third text, the method further includes: Based on the third text and the prosodic tag of each word in the third text, speech synthesis is performed on the third text to obtain speech corresponding to the third text.

8. The method according to any one of claims 1 to 7, It is characterized in that The prosody tag includes at least a first tag and a second tag, wherein the first tag indicates that there is no pause between the word and the next word of the word, and the second tag indicates that there is a pause between the word and the next word of the word, and the length of the pause between the word and the next word of the word.

9. The method according to any one of claims 1 to 7, It is characterized in that The first recognition network is a transformer-based bidirectional encoder representation BERT network.

10. A text processing device, It is characterized in that The device comprises: A labeling model acquisition module, used to acquire a prosody labeling model, wherein the prosody labeling model includes a first recognition network and a labeling network, wherein the first recognition network is a pre-trained network, the first recognition network has a function of recognizing each word in any text, and the labeling network is used to train and label a prosody label of at least one word in any text; A first training module is used to train the prosody annotation model based on a plurality of first sample data, so that the first recognition network has a function of identifying the association relationship between each two adjacent words in the text by processing the input text, and enables the annotation network to determine the prosody label of each word based on the association relationship between each two adjacent words identified by the first recognition network, wherein the first sample data includes a first text and the prosody label of each word in the first text, and the association relationship between two adjacent words at least indicates whether there is a pause between the two words; a labeling module, configured to call the trained prosody labeling model to label the plurality of second texts respectively to obtain a plurality of second sample data, wherein the second sample data includes the second text and the prosody label of each word in the second text, and the number of the plurality of first sample data is less than the number of the plurality of second sample data; The second training module is used to train a rhythm prediction model based on the multiple second sample data. The trained rhythm prediction model includes a second recognition network and a rhythm prediction network. The second recognition network has the function of identifying each word in any text and the association relationship between every two adjacent words. The rhythm prediction network has the function of predicting the rhythm label of at least one word in any text according to the recognition result of the second recognition network.

11. A computer device, It is characterized in that The computer device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed in the text processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor to implement the operations performed in the text processing method according to any one of claims 1 to 9.