Rhythm model construction method and device, rhythm labeling method, and electronic device

Abstract

The application provides a prosody model construction method and device, a prosody labeling method, an electronic device and a readable storage medium. The construction method comprises the following steps: obtaining a prediction prompt of input data, wherein the prediction prompt is used to indicate a prosody level when prosody labeling is performed on the input data; based on the prediction prompt, prosody labeling is performed on the input data to obtain prosody labeled text with the prosody level in the prediction prompt; and according to a labeling error between the prosody labeled text and an expected labeled text of the input data, a parameter of a classifier is adjusted to obtain a prosody model with the labeling error in an expected error range. The application overcomes the dependence on the completeness of the expected labeled text in the input data, and reduces the difficulty of obtaining the input data.

Description

Technical Field

[0001] This application relates to the field of intelligent speech technology, and in particular to a method for constructing a prosody model, a device for constructing a prosody model, a prosody annotation method, an electronic device, and a readable storage medium. Background Technology

[0002] Accurate control of the prosodic level of text is crucial for improving the naturalness of speech synthesis. Therefore, in the process of speech synthesis, prosodic models are typically used to analyze the prosodic level of the text to determine its prosodic structure. Clearly, prosodic models are essential in the process of obtaining prosodic levels. In related technologies, complete prosodic joint data is usually used as training samples for training prosodic models.

[0003] However, joint prosodic data requires complete annotation of each prosodic level of the samples, which relies heavily on rigorous expert experience and text processing effects, making it difficult to obtain. Decoupled prosodic data corresponding to each prosodic level is relatively easy to obtain, but if multiple prosodic models are built using multiple decoupled prosodic data, the parameters of each prosodic model cannot be shared, and flexible prosodic level training cannot be supported. Summary of the Invention

[0004] To address at least one of the aforementioned technical problems, this application provides a method for constructing a prosody model, an apparatus for constructing a prosody model, a prosody annotation method, an electronic device, and a readable storage medium.

[0005] One aspect of this application provides a method for constructing a prosodic model, which may include: obtaining a prediction cue for input data, wherein the prediction cue is used to indicate the prosodic level when performing prosodic annotation on the input data; performing prosodic annotation on the input data based on the prediction cue to obtain prosodic-annotated text with the prosodic level in the prediction cue; and adjusting the parameters of a classifier according to the annotation error between the prosodic-annotated text and the expected annotation text of the input data to obtain a prosodic model that makes the annotation error within the expected error range.

[0006] In some implementations, prosodic annotation of input data based on predictive prompts to obtain prosodic-annotated text with prosodic levels from the predictive prompts may include: extracting prosodic levels for the input data from the predictive prompts, wherein prosodic levels include prosodic words, prosodic phrases, and intonation phrases; activating annotation channels corresponding to the prosodic levels; and using the annotation channels to perform prosodic annotation on the training text of the input data to obtain prosodic-annotated text with prosodic levels.

[0007] In some implementations, before obtaining prediction prompts for the input data, the process may include: filtering input data from a sample library; extracting sample library features from the sample library; determining the input method for each sample data in the sample library based on the sample library features; and determining the input data from multiple sample data based on the input method.

[0008] In some implementations, the input method for each sample data in the sample library is determined based on the characteristics of the sample library. This may include: when the number of sample data corresponding to each prosodic level is similar, the input method is to input sample data with the same prosodic level one by one; or when the number of sample data corresponding to each prosodic level differs from a preset threshold, the input method is to input sample data corresponding to different prosodic levels in a cross-cutting manner.

[0009] In some implementations, adjusting the parameters of a classifier based on the annotation error between the prosodic annotation text and the expected annotation text of the input data to obtain a prosodic model that keeps the annotation error within the expected error range may include: activating at least one annotation channel in the classifier corresponding to the input mode, and adjusting the parameters of at least one annotation channel based on the annotation error between the prosodic annotation text and the expected annotation text of the input data to obtain a prosodic model that keeps the annotation error within the expected error range.

[0010] In some implementations, at least one annotation channel corresponding to the input method is activated in the classifier, and the parameters of the at least one annotation channel are adjusted according to the annotation error between the prosodic annotation text and the expected annotation text of the input data to obtain a prosodic model in which the annotation error is within the expected error range. This may include: in response to an input method that sequentially inputs sample data with the same prosodic level, activating the annotation channel corresponding to the input sample in the classifier, and adjusting the parameters of the annotation channel corresponding to the input sample according to the annotation error to obtain a prosodic model in which the annotation error is within the expected error range; or in response to an input method that cross-inputs sample data corresponding to different prosodic levels, activating all annotation channels in the classifier, and adjusting the parameters of each annotation channel according to the annotation error to obtain a prosodic model in which the annotation error is within the expected error range.

[0011] In some implementations, before performing prosodic annotation on the input data based on the predicted prompts to obtain prosodic-annotated text with the prosodic level in the predicted prompts, the method may further include: performing feature extraction on the input data to obtain a feature vector for characterizing the text features of the input data.

[0012] Another aspect of this application provides an apparatus for constructing a prosodic model, which may include: an acquisition module for acquiring a prediction cue for input data, wherein the prediction cue indicates the prosodic level when performing prosodic annotation on the input data; an annotation module for performing prosodic annotation on the input data based on the prediction cue, obtaining prosodic-annotated text with the prosodic level in the prediction cue; and an adjustment module for adjusting the parameters of a classifier according to the annotation error between the prosodic-annotated text and the expected annotated text of the input data, so as to obtain a prosodic model that makes the annotation error within a desired error range.

[0013] Another aspect of this application provides a method for obtaining prosodic annotation text, which may include: obtaining target text; performing prosodic annotation on the target text using a prosodic model, wherein the prosodic model is constructed by any of the prosodic model construction methods described in the above embodiments; and generating prosodic annotation text of the target text, wherein the prosodic annotation text has annotation information corresponding to at least one prosodic level.

[0014] Another aspect of this application provides a prosodic text acquisition apparatus, which may include: a target text acquisition module for acquiring target text; a target annotation module for prosodic annotation of the target text using a prosodic model, wherein the prosodic model is constructed by any of the prosodic model construction methods described in the above embodiments; and a result generation module for generating prosodic annotated text of the target text, wherein the prosodic annotated text has annotation information corresponding to at least one prosodic level.

[0015] Another aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the method for constructing a prosody model as described in any of the above embodiments.

[0016] Another aspect of this application provides a readable storage medium storing a computer program adapted for loading by a processor to perform steps in the method for constructing a prosody model as described in any of the above embodiments.

[0017] According to the prosodic model construction method and apparatus, prosodic annotation method, electronic device, and readable storage medium of this application, the input data is prosodicly annotated according to the prosodic level indicated in the prediction prompt. Then, the classifier parameters are adjusted based on the annotation error between the prosodic annotated text and the expected annotated text, ultimately obtaining a prosodic model where the annotation error is within the expected error range. This application overcomes the dependence on the completeness of the expected annotated text in the input data and reduces the difficulty of obtaining the input data. Attached Figure Description

[0018] The accompanying drawings illustrate exemplary embodiments of the present application and, together with the description thereof, serve to explain the principles of the present application. These drawings are included to provide a further understanding of the present application and are incorporated in and constitute a part of this specification.

[0019] Figure 1 This is a flowchart illustrating a method for constructing a prosody model according to an exemplary embodiment of this application;

[0020] Figure 2 Here is a complete training flowchart of a prosody model according to an exemplary embodiment of this application;

[0021] Figure 3 This is a schematic diagram illustrating the sequential input according to an exemplary embodiment of this application;

[0022] Figure 4 This is a schematic diagram of cross-input according to an exemplary embodiment of this application;

[0023] Figure 5 This is a block diagram of a prosody model construction apparatus according to an exemplary embodiment of this application;

[0024] Figure 6 A flowchart illustrating a prosodic annotation method according to an exemplary embodiment of this application; and

[0025] Figure 7 This is a block diagram of a prosody annotation device according to an exemplary embodiment of this application. Detailed Implementation

[0026] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present application are shown in the accompanying drawings.

[0027] It should be noted that, where there is no conflict, the embodiments and features described in this application can be combined with each other. The technical solutions of this application will now be described in detail with reference to the accompanying drawings and embodiments.

[0028] Unless otherwise stated, the exemplary implementations / embodiments shown are to be understood as providing exemplary features of various details that provide ways to implement the technical concept of this application in practice. Therefore, unless otherwise stated, the features of various implementations / embodiments may be additionally combined, separated, interchanged and / or rearranged without departing from the technical concept of this application.

[0029] The terminology used herein is for the purpose of describing particular embodiments and is not restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms “a” and “the” are intended to include the plural forms as well. Furthermore, when the terms “comprising” and / or “including” and variations thereof are used in this specification, it indicates the presence of the stated features, integrals, steps, operations, parts, components, and / or groups thereof, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, parts, components, and / or groups thereof. It should also be noted that, as used herein, the terms “substantially,” “about,” and other similar terms are used as approximate terms rather than as terms of degree, thus explaining the inherent biases in measurements, calculated values, and / or provided values ​​that would be recognized by one of ordinary skill in the art.

[0030] Figure 1 This is a flowchart illustrating a method for constructing a prosody model according to an exemplary embodiment of this application. Figure 1 As shown, the prosodic model construction method M100 may include: Step S102, obtaining prediction prompts for the input data, wherein the prediction prompts are used to indicate the prosodic level when performing prosodic annotation on the input data. Step S104, performing prosodic annotation on the input data based on the prediction prompts to obtain prosodic-annotated text with the prosodic level in the prediction prompts. Step S106, adjusting the parameters of the classifier according to the annotation error between the prosodic-annotated text and the expected annotation text of the input data to obtain a prosodic model that makes the annotation error within the expected error range.

[0031] Figure 2 This is a complete training flowchart of a prosodic model according to an exemplary embodiment of this application. The following will combine... Figure 2 The specific implementation methods of each step in the above construction method M100 are explained in detail.

[0032] Step S102: Obtain prediction prompts for the input data, wherein the prediction prompts are used to indicate the prosodic level when performing prosodic annotation on the input data.

[0033] The input data includes prediction prompts, training text, and expected labeled text. In this application, a Layer Look-Up Table (LLUT) is used to identify prediction prompts in the input data. These prediction prompts indicate the prosodic level when performing prosodic annotation on the training text in the input data. Due to the LLUT configuration, the classifier can respond to prosodic annotations of various prosodic levels, providing a more universal space for input data selection and making the construction process of the prosodic model in this application more flexible.

[0034] Prosodic levels include prosodic words (PW), prosodic phrases (PPH), and intonation phrases (IPH). Prosodic levels characterize the prosodic structure of the training text. During speech synthesis, pauses can be added at appropriate points based on the prosodic level annotations in the training text, making the synthesized speech sound more natural and closer to actual pronunciation. The expected annotated text can be decoupled prosodic annotation text with only a single prosodic level, reducing the difficulty of obtaining input data.

[0035] Specifically, prosodic words, defined from a prosodic perspective, refer to the smallest freely usable linguistic unit. In other words, a prosodic word is a group of syllables that are closely related and frequently pronounced together in actual language communication. A prosodic phrase is a medium-rhythmic block between a prosodic word and an intonation phrase. Rhythmic boundaries may exist between the individual prosodic words within a phrase, and it possesses a relatively stable phrase intonation pattern and phrase stress configuration. An intonation phrase is formed by connecting multiple prosodic phrases according to a certain intonation pattern, corresponding to a sentence in syntax.

[0036] For example, if the training text is “Using the program saved time and improved the accuracy of the data”, then the prosodic words could be “using”, “program”, “save”, “time”, “and”, “improved”, “data”, and “accuracy”; the prosodic phrases could be “Using the program saved time” and “and improved the accuracy of the data”; and the intonation phrases could be “Using the program saved time and improved the accuracy of the data”.

[0037] In some implementations, before obtaining prediction prompts for the input data, the process includes filtering the input data from a sample library. Specifically, this can be achieved by: extracting sample library features; determining the input method for each sample data point in the sample library based on these features; and determining the input data from multiple sample data points based on the input method.

[0038] The sample database contains multiple sample datasets, each including prediction prompts, training text, and expected labeled text. The input data is actually sample data selected from the sample database based on the input method.

[0039] For the sample database, each sample data is independent, and the content of the training text for each sample data does not need to be the same. For example, in sample data A, the prediction prompt can be the prosodic annotation of prosodic words in training text a; in sample data B, the prediction prompt can be the prosodic annotation of prosodic phrases in training text b, and the text content of training text a and training text b does not need to be the same. In this way, any existing training data can be used as sample data, avoiding the difficulty of obtaining sample data.

[0040] Typically, the number of sample data points falling into each prosodic level in the sample library is counted, and the sample library characteristics are determined based on the ratio of the number of sample data points corresponding to each prosodic level. Sample library characteristics may include similar numbers of sample data points corresponding to each prosodic level, or differences in the number of sample data points corresponding to each prosodic level exceeding a preset threshold. The preset threshold can be pre-set according to actual needs, and its specific value is not limited here.

[0041] After obtaining the features of each sample library, different input methods for sample data can be matched according to the different features of the sample library.

[0042] Figure 3 This is a schematic diagram illustrating the sequential input according to an exemplary embodiment of this application. For example... Figure 3 The diagram illustrates the training process when the number of sample data corresponding to each prosodic level in the sample library is similar. The input method involves sequentially inputting sample data with the same prosodic level (i.e., Flatten training). That is, after training all sample data corresponding to one prosodic level, all sample data corresponding to another prosodic level are input. Solid arrows indicate the working state, and dashed arrows indicate the state awaiting input.

[0043] For example, when the number of samples corresponding to prosodic words, prosodic phrases, and intonation phrases in the sample library is the same or similar, all sample data corresponding to the prosodic word PW can be input first. After passing through the data loader, the labeled channels corresponding to the prosodic word are trained. At this time, the sample data corresponding to the prosodic phrase PPH and the sample data corresponding to the intonation phrase IPH are in a pending input state. The input instructions should be executed sequentially only after all the sample data corresponding to the prosodic word has been input and the corresponding labeled channels have been trained. Of course, the input order of the sample data for each prosodic level can be changed without restriction. This input method can meet the situation where the sample data corresponding to various prosodic levels differ greatly. That is, all sample data corresponding to the same prosodic level are input sequentially, the labeled channels corresponding to that prosodic level are trained intensively, and after the parameters of the labeled channels corresponding to that prosodic level are fixed, all sample data for the next prosodic level are input without interfering with the parameters of other labeled channels.

[0044] Figure 4 This is a schematic diagram of a cross-input method according to an exemplary embodiment of this application. For example... Figure 4As shown, when the difference in the number of sample data corresponding to each prosodic level in the sample library exceeds a preset threshold, the input method is to cross-input sample data corresponding to different prosodic levels (i.e., cross training method). This can shuffle the input order of sample data corresponding to each prosodic level, eliminating the need to input sample data of one type of prosodic level before inputting sample data of another type of prosodic level.

[0045] For example, when the ratio of the number of samples corresponding to prosodic words, prosodic phrases, and intonation phrases in the sample library exceeds a preset threshold, sample data corresponding to a prosodic word, a prosodic phrase, and an intonation phrase can be input sequentially and intermittently. This input method can accommodate situations where the sample data corresponding to various prosodic levels are relatively similar. By intermittently inputting sample data corresponding to each prosodic level, and then fine-tuning the parameters of the annotation channels for the remaining prosodic levels based on the current input data, the parameters of each annotation channel are made more adaptable to real-world scenarios.

[0046] Step S104: Based on the prediction prompts, perform prosodic annotation on the input data to obtain prosodic annotated text with the prosodic level in the prediction prompts.

[0047] The classifier has multiple annotation channels, and the corresponding annotation channels can be activated for different prosody levels in the prediction prompts.

[0048] Specifically, prosodic levels for the input data are extracted from the prediction prompts, where prosodic levels include prosodic words, prosodic phrases, and intonation phrases. Based on the prosodic levels, annotation channels corresponding to those levels are activated. Using these annotation channels, the training text of the input data is prosodically annotated to obtain prosodic-annotated text with prosodic levels.

[0049] For example, when the input data is "Using the program saved time and improved data accuracy," if the prediction prompt instructs that prosodic words be annotated, the prosodic annotation text could be "Using program #1 saved #1 time and improved #1 data accuracy #1," where "#1" represents the annotation of a prosodic word. If the prediction prompt instructs that prosodic phrases be annotated, the prosodic annotation text could be "Using the program saved #2 time and improved data accuracy #2," where "#2" represents the annotation of a prosodic phrase. If the prediction prompt instructs that intonation phrases be annotated, the prosodic annotation text could be "Using the program saved time and improved data accuracy #3," where "#3" represents the annotation of a prosodic phrase.

[0050] It should be noted that this application has only one classifier, which provides multiple prosodic level annotation channels. The input data can be mapped to the corresponding prosodic level annotation channels, thereby realizing information sharing between different prosodic levels and avoiding the cumbersome model structure in related technologies.

[0051] Step S106: Adjust the parameters of the classifier based on the annotation error between the prosodic annotation text and the expected annotation text of the input data to obtain a prosodic model that makes the annotation error within the expected error range.

[0052] Specifically, the prosodic annotation text is compared with the expected annotation text of the input data to obtain the annotation error between the two. When the annotation error is within the expected error range, it proves that the prosodic annotation text output by the current classifier is relatively accurate, and the prosodic model containing this classifier is usable. If the annotation error is outside the expected error range, it proves that the prosodic annotation text output by the current classifier has a large deviation. If this classifier is used to output prosodic annotation text, the pauses in the synthesized speech will deviate from the real speaking scenario. In this case, the parameters of the annotation channel in the classifier corresponding to the current prosodic level should be adjusted according to the annotation error until the annotation error between the output prosodic annotation text and the expected annotation text of the input data is within the expected error range.

[0053] More specifically, at least one annotation channel corresponding to the input method is activated in the classifier. Based on the annotation error between the prosodic annotation text and the expected annotation text of the input data, the parameters of at least one annotation channel are adjusted to obtain a prosodic model where the annotation error is within the expected error range. For example, in response to an input method that sequentially inputs sample data with the same prosodic level, the annotation channel corresponding to the input sample is activated in the classifier. Based on the annotation error, the parameters of the annotation channel corresponding to the input sample are adjusted to obtain a prosodic model where the annotation error is within the expected error range, ultimately outputting prosodic annotation text that meets the expectations. In response to an input method that cross-inputs sample data corresponding to different prosodic levels, all annotation channels are activated in the classifier. Based on the annotation error, the parameters of each annotation channel are adjusted to obtain a prosodic model where the annotation error is within the expected error range, ultimately outputting prosodic annotation text that meets the expectations.

[0054] In some implementations, before performing prosodic annotation on the input data based on the predicted prompts to obtain prosodic-annotated text with the prosodic level in the predicted prompts, the method further includes: performing feature extraction on the input data to obtain a feature vector used to characterize the text features of the input data.

[0055] The feature vector includes structural and semantic information of the training text in the input data.

[0056] The semantic modeling unit is primarily used to perform NLP (Natural Language Processing), including syntactic analysis, discourse understanding, and semantic analysis. In this application, the semantic modeling unit is used to extract feature vectors from the training text of the input data. The semantic modeling unit can be BERT (Bidirectional Encoder Representation from Transformers, a pre-trained language representation model) or GPT (Generative Pre-Training, a pre-trained model) for NLP, or other models capable of extracting feature vectors from the input data; there are no further restrictions. The semantic modeling unit has a 3D 128-dimensional embedding layer, which can pass input data of different prosodic levels to the corresponding labeled channels in the classifier, improving the accuracy of prosodic labeled text.

[0057] It should be noted that, in Figure 2 In the standard approach, feature vectors of the input data are extracted first, followed by prediction prompts. However, in practical applications, feature vector extraction and prediction prompt acquisition can occur simultaneously or in any order; no restriction is placed here. Finally, the feature vectors and prediction prompts are input into the classifier to support prosodic annotation and obtain prosodic-annotated text.

[0058] According to the prosodic model construction method of this application, the input data is prosodicly labeled according to the prosodic level indicated in the prediction prompt. Then, the parameters of the classifier are adjusted according to the labeling error between the prosodic labeled text and the expected labeled text, and finally a prosodic model is obtained that makes the labeling error within the expected error range. This application overcomes the dependence on the completeness of the expected labeled text in the input data and reduces the difficulty of obtaining input data.

[0059] Figure 5 This is a block diagram of a prosody model construction apparatus according to an exemplary embodiment of this application. Figure 5 As shown, this application proposes a prosodic model construction apparatus 1000, comprising: an acquisition module 1002 for acquiring prediction prompts for input data, wherein the prediction prompts indicate the prosodic level when performing prosodic annotation on the input data; an annotation module 1004 for performing prosodic annotation on the input data based on the prediction prompts to obtain prosodic-annotated text with the prosodic level in the prediction prompts; and an adjustment module 1006 for adjusting the parameters of a classifier based on the annotation error between the prosodic-annotated text and the expected annotation text of the input data to obtain a prosodic model in which the annotation error is within the expected error range.

[0060] In some implementations, the execution steps of the annotation module 1004 may include: extracting prosodic levels for the input data from the prediction prompts, wherein the prosodic levels include prosodic words, prosodic phrases, and intonation phrases; activating an annotation channel corresponding to the prosodic level based on the prosodic level; and using the annotation channel to perform prosodic annotation on the training text of the input data to obtain prosodic-annotated text with prosodic levels.

[0061] In some implementations, an input data filtering module (not shown) may also be included to filter input data from the sample library; the execution steps of the module may include: extracting sample library features from the sample library; determining the input method of each sample data in the sample library based on the sample library features; and determining the input data from multiple sample data based on the input method.

[0062] In some implementations, the input method for each sample data in the sample library is determined based on the characteristics of the sample library. This may include: when the number of sample data corresponding to each prosodic level is similar, the input method is to input sample data with the same prosodic level one by one; or when the number of sample data corresponding to each prosodic level differs from a preset threshold, the input method is to input sample data corresponding to different prosodic levels in a cross-cutting manner.

[0063] In some implementations, the execution steps of the adjustment module 1006 may include: activating at least one annotation channel in the classifier corresponding to the input mode, and adjusting the parameters of the at least one annotation channel according to the annotation error between the prosodic annotation text and the expected annotation text of the input data, so as to obtain a prosodic model that makes the annotation error within the expected error range.

[0064] In some embodiments, the execution steps of the adjustment module 1006 may further include: in response to an input method that sequentially inputs sample data with the same prosody level, activating the annotation channel corresponding to the input sample in the classifier, and adjusting the parameters of the annotation channel corresponding to the input sample according to the annotation error to obtain a prosody model that makes the annotation error within the expected error range; or in response to an input method that cross-inputs sample data corresponding to different prosody levels, activating all annotation channels in the classifier, and adjusting the parameters of each annotation channel according to the annotation error to obtain a prosody model that makes the annotation error within the expected error range.

[0065] In some implementations, a feature vector extraction module (not shown) may also be included for extracting features from the input data to obtain feature vectors that characterize the textual features of the input data.

[0066] The apparatus may include corresponding modules that perform one or more steps in the flowchart above. Therefore, each or more steps in the flowchart above can be performed by a corresponding module, and the apparatus may include one or more of these modules. A module may be one or more hardware modules specifically configured to perform a corresponding step, or implemented by a processor configured to perform a corresponding step, or stored in a computer-readable medium for implementation by a processor, or implemented through some combination thereof.

[0067] This hardware architecture can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application and overall design constraints of the hardware. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0068] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this diagram, but this does not imply that there is only one bus or only one type of bus.

[0069] According to the prosodic model construction apparatus of this application, the input data is prosodicly labeled according to the prosodic level indicated in the prediction prompt. Then, the parameters of the classifier are adjusted according to the labeling error between the prosodic labeled text and the expected labeled text, and finally a prosodic model is obtained that makes the labeling error within the expected error range. This application overcomes the dependence on the completeness of the expected labeled text in the input data and reduces the difficulty of obtaining the input data.

[0070] Figure 6 This is a flowchart of a prosody annotation method according to an exemplary embodiment of this application. Figure 6 As shown, this application proposes a prosodic annotation method M200, which may include: step S202, obtaining target text; step S204, performing prosodic annotation on the target text using a prosodic model, wherein the prosodic model is constructed by the prosodic model construction method described in any of the above embodiments; step S206, generating prosodic-annotated text of the target text, wherein the prosodic-annotated text has annotation information corresponding to at least one prosodic level.

[0071] The target text is the text used for speech synthesis when dealing with real-world dialogue scenarios, and it can typically be a sentence.

[0072] The method for constructing a prosodic model may include at least the following steps: obtaining a prediction cue for the input data, wherein the prediction cue is used to indicate the prosodic level when performing prosodic annotation on the input data; performing prosodic annotation on the input data based on the prediction cue to obtain prosodic annotated text with the prosodic level in the prediction cue; and adjusting the parameters of the classifier according to the annotation error between the prosodic annotated text and the expected annotated text of the input data to obtain a prosodic model that makes the annotation error within the expected error range.

[0073] Figure 7 This is a block diagram of a prosody annotation device according to an exemplary embodiment of this application. Figure 7 As shown, this application proposes a prosodic annotation device 2000, which may include: a target text acquisition module 2002 for acquiring target text; a target annotation module 2004 for performing prosodic annotation on the target text using a prosodic model, wherein the prosodic model is constructed by the prosodic model construction method described in any of the above embodiments; and a result generation module 2006 for generating prosodic-annotated text of the target text, wherein the prosodic-annotated text has annotation information corresponding to at least one prosodic level.

[0074] The target text is the text used for speech synthesis when dealing with real-world dialogue scenarios, and it can typically be a sentence.

[0075] The method for constructing a prosodic model may include at least the following steps: obtaining a prediction cue for the input data, wherein the prediction cue is used to indicate the prosodic level when performing prosodic annotation on the input data; performing prosodic annotation on the input data based on the prediction cue to obtain prosodic annotated text with the prosodic level in the prediction cue; and adjusting the parameters of the classifier according to the annotation error between the prosodic annotated text and the expected annotated text of the input data to obtain a prosodic model that makes the annotation error within the expected error range.

[0076] Any process or method described in the flowcharts or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which the embodiments of this application pertain. The processor performs the various methods and processes described above. For example, the method embodiments of this application can be implemented as software programs tangibly contained in a machine-readable medium, such as memory. In some embodiments, part or all of the software program may be loaded and / or installed via memory and / or a communication interface. When the software program is loaded into memory and executed by the processor, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the processor may be configured to perform one of the methods described above by any other suitable means (e.g., by means of firmware).

[0077] The logic and / or steps represented in the flowchart or otherwise described herein may be specifically implemented in any readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0078] For the purposes of this specification, a "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM). Furthermore, a readable storage medium can even be paper or other suitable media on which a program can be printed, since a program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in memory.

[0079] It should be understood that various parts of this application can be implemented in hardware, software, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0080] Those skilled in the art will understand that all or part of the steps of the methods described above can be implemented by a program instructing related hardware, and the program can be stored in a readable storage medium. When executed, the program includes one or a combination of the steps of the method implementation.

[0081] Furthermore, the functional units in the various embodiments of this application can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a readable storage medium. The storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0082] This application also provides an electronic device, including: a memory storing execution instructions; and a processor or other hardware module executing the execution instructions stored in the memory, causing the processor or other hardware module to perform the above-described method.

[0083] This application also provides a readable storage medium storing execution instructions, which, when executed by a processor, are used to implement a method for constructing a prosodic model, including: obtaining a prediction prompt for input data, wherein the prediction prompt is used to indicate the prosodic level when performing prosodic annotation on the input data; performing prosodic annotation on the input data based on the prediction prompt to obtain prosodic-annotated text with the prosodic level in the prediction prompt; and adjusting the parameters of a classifier according to the annotation error between the prosodic-annotated text and the expected annotation text of the input data to obtain a prosodic model that makes the annotation error within the expected error range.

[0084] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment / mode or example is included in at least one embodiment / mode or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0085] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0086] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. A method for constructing a prosodic model, characterized in that, include: Obtain prediction prompts for the input data, the prediction prompts being used to indicate the prosodic level when performing prosodic annotation on the input data; Based on the prediction prompts, the input data is subjected to prosodic annotation to obtain prosodic annotated text with the prosodic level in the prediction prompts; as well as Based on the annotation error between the prosodic annotation text and the expected annotation text of the input data, the parameters of the classifier are adjusted to obtain a prosodic model that makes the annotation error within the expected error range. The step of performing prosodic annotation on the input data based on the predicted prompt to obtain prosodic-annotated text with the prosodic level in the predicted prompt includes: Extract the prosodic level for the input data from the prediction prompts, wherein the prosodic level includes prosodic words, prosodic phrases and intonation phrases; Based on the prosodic level, activate the annotation channel corresponding to the prosodic level; and Using the annotation channel, the training text of the input data is subjected to prosodic annotation to obtain prosodic annotated text with the prosodic level; The step of adjusting the classifier parameters based on the annotation error between the prosodic annotation text and the expected annotation text of the input data to obtain a prosodic model that makes the annotation error within the expected error range includes: In the classifier, at least one annotation channel corresponding to the input mode of the input data is activated. Based on the annotation error between the prosodic annotation text and the expected annotation text of the input data, the parameters of at least one annotation channel are adjusted to obtain a prosodic model that makes the annotation error within the expected error range.

2. The method for constructing a prosodic model according to claim 1, characterized in that, Before obtaining the prediction prompts for the input data, the following are included: The input data is selected from the sample library, including: Extract the sample library features from the sample library; Based on the characteristics of the sample library, determine the input method for each sample data in the sample library; Based on the input method, the input data is determined from multiple sample data.

3. The method for constructing a prosodic model according to claim 2, characterized in that, The step of determining the input method for each sample data in the sample library based on the characteristics of the sample library includes: When the sample library features a similar number of sample data corresponding to each prosodic level, the input method is to input sample data with the same prosodic level one by one; or When the difference in the number of sample data corresponding to each prosodic level exceeds a preset threshold, the input method is to cross-input sample data corresponding to different prosodic levels.

4. The method for constructing a prosodic model according to claim 1, characterized in that, The step of activating at least one annotation channel corresponding to the input mode in the classifier, and adjusting the parameters of at least one annotation channel based on the annotation error between the prosodic annotation text and the expected annotation text of the input data to obtain a prosodic model that makes the annotation error within the expected error range, includes: In response to the input method of sequentially inputting sample data with the same prosodic level, a labeled channel corresponding to the input data is activated in the classifier. Based on the labeled error, the parameters of the labeled channel corresponding to the input data are adjusted to obtain a prosodic model that keeps the labeled error within a desired error range; or In response to the input method of cross-inputting sample data corresponding to different prosodic levels, all labeled channels are activated in the classifier, and the parameters of each labeled channel are adjusted according to the labeled error to obtain a prosodic model that makes the labeled error within the expected error range.

5. A device for constructing a rhythmic model, characterized in that, include: The acquisition module is used to acquire prediction prompts for input data, wherein the prediction prompts are used to indicate the prosodic level when performing prosodic annotation on the input data; The annotation module is used to perform prosodic annotation on the input data based on the prediction prompt, and obtain prosodic annotation text with the prosodic level in the prediction prompt; as well as An adjustment module is used to adjust the parameters of the classifier based on the annotation error between the prosodic annotation text and the expected annotation text of the input data, so as to obtain a prosodic model that makes the annotation error within the expected error range. The step of performing prosodic annotation on the input data based on the predicted prompt to obtain prosodic-annotated text with the prosodic level in the predicted prompt includes: Extract the prosodic level for the input data from the prediction prompts, wherein the prosodic level includes prosodic words, prosodic phrases and intonation phrases; Based on the prosodic level, activate the annotation channel corresponding to the prosodic level; and Using the annotation channel, the training text of the input data is subjected to prosodic annotation to obtain prosodic annotated text with the prosodic level; The step of adjusting the classifier parameters based on the annotation error between the prosodic annotation text and the expected annotation text of the input data to obtain a prosodic model that makes the annotation error within the expected error range includes: In the classifier, at least one annotation channel corresponding to the input mode of the input data is activated. Based on the annotation error between the prosodic annotation text and the expected annotation text of the input data, the parameters of at least one annotation channel are adjusted to obtain a prosodic model that makes the annotation error within the expected error range.

6. A method for prosodic annotation, characterized in that, include: Get the target text; The target text is prosodicly annotated using a prosodic model, wherein the prosodic model is constructed by the method for constructing a prosodic model as described in any one of claims 1 to 4; and Generate prosodic text for the target text, wherein the prosodic text has annotation information corresponding to at least one prosodic level.

7. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for constructing a prosody model as described in any one of claims 1 to 4.

8. A readable storage medium, characterized in that, The readable storage medium stores a computer program adapted for loading by a processor to execute the method for constructing a prosody model as described in any one of claims 1 to 4.

Images