Speech text generation method, training method and device of speech text generation model

Abstract

The present disclosure provides a speech text generation method, which can be applied to the fields of artificial intelligence technology and intelligent customer service. The speech text generation method comprises the following steps: performing part-of-speech tagging on a standard text to obtain a part-of-speech tagging result; determining a target part of speech from the part-of-speech tagging result according to a distribution feature of emotional words; determining a predicted insertion position according to the position of the content corresponding to the target part of speech in the standard text; inserting a target emotional word into the standard text according to the predicted insertion position to obtain a target spoken language text; and generating a target speech text according to the target spoken language text. The present disclosure also provides a training method of a speech text generation model, a speech text generation device, a training device of a speech text generation model, equipment, a medium and a program product.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence technology and intelligent customer service technology, and more specifically, to a speech-to-text generation method, a training method for a speech-to-text generation model, an apparatus, a device, a medium, and a program product. Background Technology

[0002] With the development of artificial intelligence technology, the application scenarios of intelligent dialogue systems are becoming more and more widespread. Intelligent dialogue systems automatically generate intelligent voice information related to business or generate intelligent response information based on the user's voice information, thereby realizing automatic voice interaction with users to meet their relevant needs.

[0003] In realizing the present invention, the inventors discovered at least the following problems in the related technology: intelligent dialogue systems usually convert text with standard written sentence patterns into intelligent voice information, and the generated intelligent voice information is relatively stiff and rigid, which is quite different from the voice information generated in normal human dialogue. Summary of the Invention

[0004] In view of this, this disclosure provides a speech-to-text generation method, a training method for a speech-to-text generation model, an apparatus, a device, a medium, and a program product.

[0005] One aspect of this disclosure provides a method for generating speech-to-text, comprising:

[0006] Perform part-of-speech tagging on the standard text to obtain the part-of-speech tagging results;

[0007] The target part of speech is determined from the above part-of-speech tagging results based on the distribution characteristics of modal particles;

[0008] The predicted insertion position is determined based on the position of the content corresponding to the target part of speech in the standard text.

[0009] Based on the predicted insertion positions, the target modal particle is inserted into the standard text to obtain the target spoken text; and

[0010] Generate target speech text based on the target spoken text described above.

[0011] According to embodiments of this disclosure, the target modal particle is inserted into the standard text according to the predicted insertion position to obtain the target spoken text, including:

[0012] The predicted insertion positions of the above standard text are masked to obtain the masked standard text;

[0013] The above-mentioned masked standard text is input into the speech text generation model so that the speech text generation model inserts the target modifier at the target insertion position in the above-mentioned predicted insertion position, thereby generating the above-mentioned target spoken text.

[0014] According to embodiments of this disclosure, the above-described speech-to-text generation method further includes:

[0015] Obtain an initial corpus set, wherein the initial corpus set includes initial spoken language text generated from spoken speech corpus;

[0016] The initial spoken language corpus text was tagged with parts of speech to obtain the spoken language corpus part of speech tagging results and the spoken language corpus modal particle tagging results;

[0017] Based on the part-of-speech tagging results and the modal particle tagging results of the above spoken language corpus, the distribution characteristics of the above modal particles are determined.

[0018] According to embodiments of this disclosure, part-of-speech tagging is performed on standard text to obtain the part-of-speech tagging results, including:

[0019] The above standard text is input into the semantic recognition model to obtain the above part-of-speech tagging results;

[0020] The aforementioned semantic recognition model includes:

[0021] The first semantic recognition model is constructed based on a recurrent neural network model and a conditional random field model; or

[0022] A second semantic recognition model based on dependency parsing.

[0023] Another aspect of this disclosure provides a method for training a speech-to-text generation model, including:

[0024] Part-of-speech tagging is performed on the standard training texts and the spoken training texts associated with the standard training texts in the training sample set, respectively, to obtain the first part-of-speech tagging result of the standard training texts, the second part-of-speech tagging result of the spoken training texts, and the sample modal particle tagging result of the spoken training texts.

[0025] The target part of speech of the sample is determined from the part-of-speech tagging results of the first sample based on the distribution characteristics of the modal particles in the sample.

[0026] The sample prediction insertion position is determined based on the position of the sample content corresponding to the target part of speech of the above sample in the standard text of the above training samples.

[0027] The sample prediction insertion positions in the above training sample standard text are masked to obtain the training sample mask standard text, wherein the above training sample mask standard text has the first sample part-of-speech tagging result;

[0028] The initial speech-text generation model is trained using the target training set to obtain the trained speech-text generation model. The target training set includes the standard text masked by the training samples, the second part-of-speech tagging results of the spoken text of the training samples, and the sample tone word tagging results of the spoken text of the training samples.

[0029] According to embodiments of this disclosure, the training method for the above-described speech-to-text generation model further includes:

[0030] The first sample standard text and the first sample spoken text associated with the first sample standard text in the first sample set are updated respectively using the sample confusion words in the sample confusion dictionary, so as to obtain a second sample set containing the second sample standard text and the second sample spoken text.

[0031] The training sample set is constructed based on the first sample set and the second sample set.

[0032] According to embodiments of this disclosure, the training method for the above-described speech-to-text generation model further includes:

[0033] The standard sample text was processed using a speech synthesis device to obtain the sample speech data;

[0034] Speech recognition was performed on the above sample speech corpus to obtain the sample confused speech corpus text;

[0035] Based on the above-mentioned standard sample corpus text and the above-mentioned confused sample corpus text, the above-mentioned sample confusion dictionary is constructed.

[0036] According to embodiments of this disclosure, the training method for the above-described speech-to-text generation model further includes:

[0037] Obtain the initial sample corpus set, wherein the initial sample corpus set includes the initial sample spoken corpus text generated based on the sample spoken speech corpus;

[0038] The initial spoken language corpus text of the above samples was tagged with parts of speech to obtain the part-of-speech tagging results and the tone mark results of the spoken language corpus.

[0039] Based on the part-of-speech tagging results and the modal particle tagging results of the above-mentioned sample spoken language corpus, the distribution characteristics of the modal particles in the above-mentioned sample are determined.

[0040] Another aspect of this disclosure provides a speech-to-text generation apparatus, comprising:

[0041] The annotation module is used to perform part-of-speech tagging on standard text and obtain the part-of-speech tagging results.

[0042] The first determining module is used to determine the target part of speech from the above part-of-speech tagging results based on the distribution characteristics of modal particles;

[0043] The second determining module is used to determine the predicted insertion position based on the position of the content corresponding to the target part of speech in the standard text.

[0044] The insertion module is used to insert the target modal particle into the standard text according to the predicted insertion position, thereby obtaining the target spoken text; and

[0045] The generation module is used to generate target speech text based on the target spoken text mentioned above.

[0046] Another aspect of this disclosure provides a training apparatus for a speech-to-text generation model, comprising:

[0047] The sample annotation module is used to perform part-of-speech tagging on the standard training sample text and the spoken training sample text associated with the standard training sample text in the training sample set, respectively, to obtain the first part-of-speech tagging result of the standard training sample text, the second part-of-speech tagging result of the spoken training sample text, and the sample modal word tagging result of the spoken training sample text.

[0048] The first sample determination module is used to determine the target part of speech of the sample from the part-of-speech tagging results of the first sample based on the distribution characteristics of the modal particles in the sample.

[0049] The second sample determination module is used to determine the sample prediction insertion position based on the position of the sample content corresponding to the target part of speech of the above sample in the standard text of the above training samples;

[0050] The sample masking module is used to mask the sample prediction insertion positions in the above-mentioned training sample standard text to obtain the training sample mask standard text, wherein the above-mentioned training sample mask standard text has the first sample part-of-speech tagging result;

[0051] The training module is used to train an initial speech-text generation model using a target training set to obtain a trained speech-text generation model. The target training set includes the standard text masked by the training samples, the part-of-speech tagging results of the second samples of the spoken text of the training samples, and the tagging results of the modal particles of the spoken text of the training samples.

[0052] Another aspect of this disclosure provides an electronic device comprising:

[0053] One or more processors;

[0054] Memory, used to store one or more programs.

[0055] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.

[0056] Another aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed, are used to implement the method described above.

[0057] Another aspect of this disclosure provides a computer program product including computer-executable instructions that, when executed, are used to implement the method described above.

[0058] According to embodiments of this disclosure, after performing part-of-speech tagging on standard text and obtaining the tagging results, the part-of-speech of each standard word in the standard text can be obtained based on the tagging results. Based on this distribution feature, the predicted insertion position of the modifier can be determined from the part-of-speech standard results. The target modifier is then inserted into the standard text according to the predicted insertion position. This allows the resulting target spoken text to have the conversational characteristics of normal human dialogue. As a result, the target speech text generated from the target spoken text can at least partially solve the technical problem of the relatively stiff and rigid intelligent speech information, which is significantly different from human dialogue. This makes the target speech text closer to the conversational characteristics of human speech information, giving the target speech text a human-like technical effect and improving the user experience during voice interaction. Attached Figure Description

[0059] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0060] Figure 1 This illustration schematically shows an exemplary system architecture to which speech-to-text generation methods and apparatus can be applied according to embodiments of the present disclosure;

[0061] Figure 2 A flowchart illustrating a speech-to-text generation method according to an embodiment of the present disclosure is shown schematically.

[0062] Figure 3 A flowchart illustrating a speech-to-text generation method according to another embodiment of the present disclosure is shown schematically;

[0063] Figure 4 This schematically illustrates a flowchart of inserting a target modal particle into standard text according to a predicted insertion position, according to an embodiment of the present disclosure, to obtain target spoken text;

[0064] Figure 5 This diagram illustrates an application scenario of the speech-to-text generation method according to an embodiment of the present disclosure.

[0065] Figure 6 A flowchart illustrating a training method for a speech-to-text generation model according to an embodiment of the present disclosure is shown schematically.

[0066] Figure 7 A block diagram of a speech-to-text generation apparatus according to an embodiment of the present disclosure is shown schematically.

[0067] Figure 8 A block diagram schematically illustrates a speech-to-text generation apparatus according to embodiments of the present disclosure; and

[0068] Figure 9 A block diagram of an electronic device suitable for implementing a speech-to-text generation method and a training method for a speech-to-text generation model, according to embodiments of the present disclosure, is shown schematically. Detailed Implementation

[0069] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0070] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0071] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0072] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0073] Intelligent dialogue systems typically generate speech information based on written text, ignoring the interjections, hesitation words, and restatements that may occur in real-world human conversations. Therefore, while the speech information generated by speech synthesis devices or human transcription may be very standard, it is often stiff and unnatural, making it easy for users to perceive that they are conversing with a machine, thus degrading the user experience.

[0074] Embodiments of this disclosure provide a speech-to-text generation method, a training method for a speech-to-text generation model, an apparatus, a device, a medium, and a program product. The speech-to-text generation method includes: performing part-of-speech tagging on standard text to obtain part-of-speech tagging results; determining a target part of speech from the part-of-speech tagging results based on the distribution characteristics of modal particles; determining a predicted insertion position based on the position of the content corresponding to the target part of speech in the standard text; inserting a target modal particle into the standard text according to the predicted insertion position to obtain target spoken text; and generating target speech text based on the target spoken text.

[0075] According to embodiments of this disclosure, after performing part-of-speech tagging on standard text and obtaining the tagging results, the part-of-speech of each standard word in the standard text can be obtained based on the tagging results. Based on this distribution feature, the predicted insertion position of the modifier can be determined from the part-of-speech standard results. The target modifier is inserted into the standard text according to the predicted insertion position, which can make the obtained target spoken text have the spoken language characteristics of normal human dialogue. Thus, the target speech text generated from the target spoken text can at least partially solve the technical problem of the relatively stiff and rigid intelligent speech information and the large difference from human dialogue. It can make the target speech text closer to the spoken language characteristics of human dialogue speech information, and make the target speech text have human-like characteristics, thereby improving the user experience during voice interaction.

[0076] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0077] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0078] Figure 1 This illustration schematically depicts an exemplary system architecture to which a speech-to-text generation method and apparatus can be applied according to embodiments of this disclosure. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0079] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0080] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0081] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0082] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0083] It should be noted that the speech-to-text generation method provided in this embodiment can generally be executed by server 105. Correspondingly, the speech-to-text generation device provided in this embodiment can generally be located in server 105. The speech-to-text generation method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the speech-to-text generation device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Alternatively, the speech-to-text generation method provided in this embodiment can also be executed by terminal devices 101, 102, or 103, or by other terminal devices different from terminal devices 101, 102, or 103. Accordingly, the voice-to-text generation device provided in this embodiment may also be located in terminal devices 101, 102, or 103, or in other terminal devices different from terminal devices 101, 102, or 103.

[0084] For example, the standard text may originally be stored in any one of terminal devices 101, 102, or 103 (e.g., terminal device 101, but not limited thereto), or it may be stored on an external storage device and imported into terminal device 101. Then, terminal device 101 may execute the speech-text generation method provided in the embodiments of this disclosure locally, or send the standard text to other terminal devices, servers, or server clusters, and have the other terminal devices, servers, or server clusters that receive the standard text execute the speech-text generation method provided in the embodiments of this disclosure.

[0085] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0086] Figure 2 A flowchart illustrating a speech-text generation method according to an embodiment of the present disclosure is shown schematically.

[0087] like Figure 2 As shown, the method includes operations S210 to S250.

[0088] In operation S210, part-of-speech tagging is performed on the standard text to obtain the part-of-speech tagging results.

[0089] According to embodiments of this disclosure, standard text may include text used for written communication, such as standard customer service response text, email body text, etc.

[0090] It should be noted that standard customer service personnel response texts can be applied to intelligent customer service question and answer devices. These devices generate corresponding voice texts based on standardized customer service personnel response texts, thereby enabling voice interaction with users. However, the generated voice texts are usually rather stiff and monotonous, differing greatly from human conversational speech and lacking colloquial characteristics.

[0091] According to embodiments of this disclosure, part-of-speech tagging is performed on the tagged text, and the resulting part-of-speech tagging results may include standard words generated after word segmentation in the standard text and the part-of-speech features of the standard words. The part-of-speech features may include, for example, adjective part-of-speech, verb part-of-speech, etc.

[0092] It should be noted that the embodiments of this disclosure do not limit the specific methods of part-of-speech tagging. For example, a network model based on neural networks can be used to tag standard text with part-of-speech tags, but it is not limited to this. A semantic recognition model based on statistical algorithms can also be used to tag standard text with part-of-speech tags. The embodiments of this disclosure do not limit the specific technical means of part-of-speech tagging, and those skilled in the art can choose according to the actual situation.

[0093] In operation S220, the target part of speech is determined from the part-of-speech tagging results based on the distribution characteristics of modal particles.

[0094] According to embodiments of this disclosure, interjections may include words expressing human emotions such as hesitation or doubt, such as "um," "that's right," "for example," "similar," etc. However, this is not limited to these; it may also include polite interjections used at the beginning and / or end of human speech conversations, such as "if you have time," "if you like," etc.

[0095] According to embodiments of this disclosure, based on a large amount of dialogue text, the part of speech of words adjacent to modal particles in the dialogue text can be statistically analyzed to determine the distribution characteristics of modal particles, thereby predicting the target part of speech in the standard text based on the characteristics of modal particles.

[0096] In embodiments of this disclosure, the target part of speech may include verb and adjective parts of speech.

[0097] In operation S230, the predicted insertion position is determined based on the position of the content corresponding to the target part of speech in the standard text.

[0098] According to embodiments of this disclosure, after determining the target part of speech, the predicted insertion position can be determined based on the part-of-speech features of standard words in the standard text. The predicted insertion position can be the adjacent position of the standard word with the target part of speech.

[0099] In operation S240, the target modal particle is inserted into the standard text according to the predicted insertion position to obtain the target spoken text.

[0100] According to embodiments of this disclosure, when there are multiple predicted insertion positions, a corresponding target modifier can be inserted at each predicted insertion position, or the target modifier can be inserted at the target insertion position among the predicted insertion positions. Inserting the target modifier into the standard text enhances the colloquial characteristics of the target spoken text without altering the semantic information of the standard text.

[0101] In operation S250, target speech text is generated based on target spoken text.

[0102] According to embodiments of this disclosure, the target speech text can be speech information, and the target spoken text can be converted into target speech text using a relevant speech synthesis device, and the target speech text can be generated based on the target spoken text.

[0103] According to embodiments of this disclosure, after performing part-of-speech tagging on standard text and obtaining the tagging results, the part-of-speech of each standard word in the standard text can be obtained based on the tagging results. Based on this distribution feature, the predicted insertion position of the modifier can be determined from the part-of-speech standard results. The target modifier is then inserted into the standard text according to the predicted insertion position. This allows the resulting target spoken text to have the conversational characteristics of normal human dialogue. As a result, the target speech text generated from the target spoken text can at least partially solve the technical problem of the relatively stiff and rigid intelligent speech information, which is significantly different from human dialogue. This makes the target speech text closer to the conversational characteristics of human speech information, giving the target speech text a human-like quality and improving the user experience during voice interaction.

[0104] Figure 3 A flowchart illustrating a speech-text generation method according to another embodiment of the present disclosure is shown schematically.

[0105] like Figure 3 As shown, the speech-to-text generation method may also include operations S310 to S330.

[0106] In operation S310, an initial corpus set is obtained, which includes initial spoken corpus text generated based on spoken speech corpus.

[0107] In operation S320, part-of-speech tagging is performed on the initial spoken language corpus text to obtain the spoken language corpus part-of-speech tagging results and the spoken language corpus modal particle tagging results.

[0108] In operation S330, the distribution characteristics of modal particles are determined based on the part-of-speech tagging results and the modal particle tagging results of the spoken language corpus.

[0109] According to embodiments of this disclosure, the initial spoken language corpus text may, for example, include corpus text generated from dialogue speech information in real human conversation scenarios. This corpus text records the dialogue speech information, that is, the initial spoken language corpus text contains spoken language particles that humans habitually add. By performing part-of-speech tagging on the initial spoken language corpus text, the part-of-speech of the spoken language words in the initial spoken language corpus text can be obtained. Furthermore, the positional relationship between the spoken language particles and each spoken language word can be obtained. By analyzing and statistically analyzing this positional relationship, the distribution characteristics of the spoken language particles can be determined, that is, the distribution characteristics of particles in spoken speech information can be determined.

[0110] For example, the distribution characteristics of modal particles can be characterized by a statistical probability of 0.9 for a position after a word in spoken language corpus with the part of speech of the verb function and a statistical probability of 0.8 for a position before a word in spoken language corpus with the part of speech of the adjective function. By statistically analyzing the statistical probabilities of these positions, the distribution characteristics of modal particles can be determined.

[0111] According to embodiments of this disclosure, a position probability threshold can be set, and positions with a statistical probability threshold greater than or equal to the position probability threshold can be used as the distribution statistical probability in the distribution features of modal particles.

[0112] It should be noted that, in the embodiments of this disclosure, "before" means adjacent to the word and located before the word's position, and correspondingly, "after" means adjacent to the word and located after the word's position.

[0113] According to an embodiment of this disclosure, operation S210, which involves performing part-of-speech tagging on standard text to obtain the part-of-speech tagging result, may include the following operations.

[0114] Standard text is input into a semantic recognition model to obtain part-of-speech tagging results; the semantic recognition model includes: a first semantic recognition model based on a recurrent neural network model and a conditional random field model; or a second semantic recognition model based on dependency parsing.

[0115] According to embodiments of this disclosure, the first semantic recognition model may be constructed by sequentially connecting a recurrent neural network model (RNN model) and a conditional random field model (CRF model), or it may be constructed by sequentially connecting a bidirectional long short-term memory network model (Bi-LSTM model) and a conditional random field model (CRF model).

[0116] It should be understood that the recurrent neural network model may include the Long Short-Term Memory Network Model (LSTM model) and the Bi-LSTM model. Those skilled in the art can design the specific network structure of the first semantic recognition model according to actual needs.

[0117] According to embodiments of this disclosure, the second semantic recognition model constructed based on dependency parsing may include, for example, an LTP (Language Technology Platform) language processing system.

[0118] Figure 4 The flowchart illustrates a process of inserting a target modifier into standard text according to a predicted insertion position, based on an embodiment of the present disclosure, to obtain target spoken text.

[0119] like Figure 4 As shown, operation S240 inserts the target modifier into the standard text according to the predicted insertion position, resulting in the target spoken text, including operations S410 to S420.

[0120] In operation S410, the predicted insertion position of the standard text is masked to obtain the masked standard text.

[0121] In operation S420, the masked standard text is input into the speech text generation model so that the speech text generation model inserts the target modifier at the target insertion position in the predicted insertion position to generate the target spoken text.

[0122] According to embodiments of this disclosure, the speech text generation model can be constructed based on the BERT model, such as the BERT-WWM model. The masked standard text is input into the BERT-WWM model, which can further iterate and predict the mask of the predicted insertion position, thereby determining the target predicted position from the predicted insertion position, and determining the target tone words for each target predicted position from the tone word set based on the prediction capability of the BERT-WWM model, thereby realizing the production of target spoken text.

[0123] It should be noted that the speech-to-text generation model can be obtained after training using relevant training methods. In the case of a BERT-WWM model, the predictive ability of the BERT-WWM model can be used to replace at least some of the standard words in the masked standard text with synonyms or homophones, thereby further improving the colloquial characteristics of the target spoken text.

[0124] Figure 5 The diagram illustrates an application scenario of the speech-text generation method according to an embodiment of the present disclosure.

[0125] like Figure 5 As shown, this application scenario can include the standard text 510 "Do you need a large-size monitor?". Inputting the standard text 510 into the semantic recognition model 520 can perform part-of-speech tagging on the standard text and obtain the part-of-speech tagging result 530.

[0126] In embodiments of this disclosure, the semantic recognition model 520 may be constructed based on a sequential connection of a bidirectional long short-term memory network model (Bi-LSTM model) and a conditional random field model (CRF model).

[0127] The part-of-speech tagging result 530 can include the standard words "please," "you," "need," "large size," "monitor," and "what" from the standard text 510. It can also include the part of speech of each standard word, where "v" represents a verb, "r" represents a pronoun, "a" represents an adjective, "n" represents a noun, and "e" represents a modal particle.

[0128] According to the distribution features of modal particles, it is possible to determine that the target parts of speech are verb part of speech and adjective part of speech from the part-of-speech tagging result 530, and based on the content corresponding to the target parts of speech, determine predicted insertion positions after the verb standard word "please" and after the verb standard word "need" in the standard text 510, as well as before the adjective standard word "large-sized", and mask each predicted insertion position to obtain a masked standard text 540. The masked standard text 540 may include masking units 541 and 542 corresponding to each predicted insertion position.

[0129] The masked standard text 540 is input into a speech text generation model 550, which can determine the predicted insertion positions as target predicted insertion positions, insert the target modal particle "hmm" into the masking unit 541 corresponding to the target predicted insertion position, and insert the target modal particle "this" into the masking unit 542 corresponding to the target predicted insertion position, so as to generate the target spoken text 560 "Excuse me, hmm, do you need this large-sized monitor?". Thus, the target spoken text 560 can have colloquial characteristics close to human colloquial speech information, the target speech text generated based on the target spoken text 560 can have colloquial characteristics, at least partially avoiding the generated speech information being stiff and rigid, and reducing the difference from human conversational speech information.

[0130] Figure 6 schematically shows a flowchart of a training method for a speech text generation model according to an embodiment of the present disclosure.

[0131] As shown in Figure 6 , the method includes operations S610 to S650.

[0132] In operation S610, part-of-speech tagging is performed respectively on the training sample standard text in the training sample set and the training sample spoken text associated with the training sample standard text, to obtain a first sample part-of-speech tagging result of the training sample standard text, a second sample part-of-speech tagging result of the training sample spoken text, and a sample modal particle tagging result of the training sample spoken text.

[0133] In operation S620, a sample target part of speech is determined from the first sample part-of-speech tagging result according to the distribution features of sample modal particles.

[0134] In operation S630, a sample predicted insertion position is determined according to the position of the sample content corresponding to the sample target part of speech in the training sample standard text.

[0135] In operation S640, the sample predicted insertion position in the training sample standard text is masked to obtain a training sample masked standard text, wherein the training sample masked standard text has the first sample part-of-speech tagging result.

[0136] When operating the S650, an initial speech-to-text generation model is trained using the target training set to obtain the trained speech-to-text generation model. The target training set includes the training sample mask standard text, the second sample part-of-speech tagging results of the training sample spoken text, and the sample tone word tagging results of the training sample spoken text.

[0137] According to embodiments of this disclosure, the training sample standard text may include standard written text, and the training sample spoken text may include spoken text converted from speech information generated by a sample user's pronunciation of the training sample standard text. Since the training sample spoken text is generated based on the sample user's speech, it may contain sample interjections. This at least partially overcomes the limitations of related technologies that use standard text to train speech-to-text generation models, thus preventing the trained speech-to-text generation models from learning the characteristics of interjections, hesitation words, and other modal particles that may exist in spoken dialogue.

[0138] According to embodiments of this disclosure, training sample mask standard text and training sample spoken text can be combined to form training sample pairs. A similarity label value is determined based on the similarity between the training sample mask standard text and the training sample spoken text in the training sample pairs. This similarity label value can be used to iteratively adjust the weight parameters in the initial speech text generation model, so that the generated speech text generation model can predict the positional relationship between the sample modifier and the first sample part-of-speech tagging result in the training sample standard text. This allows the target insertion position to be accurately determined based on the predicted insertion position, and the target sample modifier to be determined from the sample modifier.

[0139] It should be noted that the speech-text generation model trained according to the training method of the speech-text generation model provided in the embodiments of this disclosure can be used in the above speech-text generation method.

[0140] According to embodiments of this disclosure, the training method for the speech-to-text generation model further includes the following operations.

[0141] The first sample standard text and the first sample spoken text associated with the first sample standard text in the first sample set are updated using the sample confusion words in the sample confusion dictionary, respectively, to obtain a second sample set containing the second sample standard text and the second sample spoken text; a training sample set is constructed based on the first sample set and the second sample set.

[0142] According to embodiments of this disclosure, the sample confusion dictionary may include sample confusion word pairs consisting of sample standard words and sample confusion words. By replacing sample standard words in the first sample standard text with sample confusion words from the sample confusion dictionary, and by replacing sample standard words in the first sample spoken text with sample confusion words from the sample confusion dictionary, a large number of second sample standard texts and second sample spoken texts can be obtained respectively. A training sample set can then be constructed based on the first and second sample sets, expanding the amount of training sample data and enhancing the training capability of the training sample set. Furthermore, by training the initial speech-text generation model using the training sample set containing the first and second sample sets, the initial speech-text generation model can fully learn the similarity relationships between standard words and confusion words. This allows the trained speech-text generation model to automatically replace standard words in the standard text with sample confusion words, thereby further enriching the semantic expression of the target spoken text and making the target spoken text closer to the colloquial characteristics of normal human conversation.

[0143] According to embodiments of this disclosure, the training method for the speech-to-text generation model may further include the following operations.

[0144] The standard sample text is processed using a speech synthesis device to obtain the sample speech text; speech recognition is performed on the sample speech text to obtain the sample confused text; and a sample confusion dictionary is constructed based on the standard sample text and the sample confused text.

[0145] According to embodiments of this disclosure, the sample standard corpus text may include text used for written communication, such as standard customer service response text, email body text, etc. The sample speech corpus may include speech information generated by a speech synthesis device after automatically recognizing the sample standard corpus text. By recognizing the sample speech corpus using an Automatic Speech Recognition (ASR) device, a recognized sample obfuscated corpus text can be obtained. Due to the limitations of the speech recognition device's recognition capabilities, at least some sample standard words in the sample standard corpus text can be recognized as sample obfuscated words, thus the sample obfuscated corpus text contains the sample obfuscated words recognized by the speech recognition device. Based on the comparison results between the sample standard corpus text and the sample obfuscated corpus text, sample standard words and sample obfuscated words can be paired to construct a sample obfuscated word dictionary.

[0146] According to embodiments of this disclosure, after performing speech recognition on sample speech corpus using a speech recognition device, the confidence information of the initial sample confusion corpus text output by the speech recognition device is determined. The initial sample confusion corpus text with confidence information less than or equal to a preset confidence threshold is determined as sample confusion corpus text. Thus, sample confusion words that are easily misidentified can be selected from the sample confusion corpus text, so that the sample confusion word pairs in the constructed sample confusion dictionary more accurately reflect the association features between sample confusion words and sample standard words.

[0147] According to embodiments of this disclosure, the training method for the speech-to-text generation model may further include the following operations.

[0148] Obtain an initial sample corpus, wherein the initial sample corpus includes initial sample spoken language text generated from sample spoken language corpus; perform part-of-speech tagging on the initial sample spoken language text to obtain part-of-speech tagging results and modal particle tagging results; determine the distribution features of the modal particles based on the part-of-speech tagging results and the modal particle tagging results.

[0149] According to embodiments of this disclosure, the initial spoken language corpus text may, for example, include corpus text generated from dialogue speech information in real human conversation scenarios. This corpus text records the dialogue speech information, that is, the initial spoken language corpus text contains spoken language particles that humans habitually add. By performing part-of-speech tagging on the initial spoken language corpus text, the part-of-speech of the sample spoken language particles in the initial spoken language corpus text can be obtained. Furthermore, the positional relationship between the sample spoken language particles and each other sample spoken language particles can be obtained. By analyzing and statistically analyzing this positional relationship, the distribution characteristics of the sample spoken language particles can be determined, that is, the distribution characteristics of sample spoken language particles in the sample spoken language speech information can be determined.

[0150] It should be noted that the speech-text generation model trained according to the training method of the speech-text generation model provided in the embodiments of this disclosure can be used in the above-described speech-text generation method.

[0151] Figure 7 A block diagram of a speech-text generation apparatus according to an embodiment of the present disclosure is shown schematically.

[0152] like Figure 7 As shown, the speech-text generation device 700 may include an annotation module 710, a first determination module 720, a second determination module 730, an insertion module 740, and a generation module 750.

[0153] The annotation module 710 is used to perform part-of-speech tagging on standard text and obtain the part-of-speech tagging results.

[0154] The first determining module 720 is used to determine the target part of speech from the part-of-speech tagging results based on the distribution characteristics of modal particles.

[0155] The second determining module 730 is used to determine the predicted insertion position based on the position of the content corresponding to the target part of speech in the standard text.

[0156] The insertion module 740 is used to insert the target modal word into the standard text according to the predicted insertion position to obtain the target spoken text.

[0157] The generation module 750 is used to generate target speech text based on the target spoken text.

[0158] According to embodiments of this disclosure, the insertion module may include a masking unit and a generation unit.

[0159] The masking unit is used to mask the predicted insertion position of the standard text to obtain the masked standard text.

[0160] The generation unit is used to input the masked standard text into the speech text generation model, so that the speech text generation model can insert the target modifier at the target insertion position in the predicted insertion position to generate the target spoken text.

[0161] According to embodiments of this disclosure, the speech-to-text generation method may further include: an acquisition module, a corpus annotation module, and a third determination module.

[0162] The acquisition module is used to acquire the initial corpus set, which includes the initial spoken corpus text generated based on the spoken speech corpus.

[0163] The corpus annotation module is used to perform part-of-speech tagging on the initial spoken language corpus text, and obtain the spoken language corpus part-of-speech tagging results and spoken language corpus mood word tagging results.

[0164] The third determination module is used to determine the distribution characteristics of modal particles based on the part-of-speech tagging results and modal particle tagging results of the spoken language corpus.

[0165] According to embodiments of this disclosure, the annotation module may include annotation units.

[0166] The annotation unit is used to input standard text into the semantic recognition model to obtain part-of-speech tagging results.

[0167] The semantic recognition model includes:

[0168] The first semantic recognition model is based on a recurrent neural network model and a conditional random field model; or the second semantic recognition model is based on dependency parsing.

[0169] It should be noted that the speech-to-text generation device part in the embodiments of this disclosure corresponds to the speech-to-text generation method part in the embodiments of this disclosure. For a detailed description of the speech-to-text generation device part, please refer to the speech-to-text generation method part, which will not be repeated here.

[0170] Figure 8 A block diagram of a speech-text generation apparatus according to an embodiment of the present disclosure is shown schematically.

[0171] like Figure 8 As shown, the speech-text generation device 800 may include a sample annotation module 810, a first sample determination module 820, a second sample determination module 830, a sample mask module 840, and a training module 850.

[0172] The sample annotation module 810 is used to perform part-of-speech tagging on the standard text of the training samples and the spoken text of the training samples associated with the standard text of the training samples, respectively, to obtain the first part-of-speech tagging result of the standard text of the training samples, the second part-of-speech tagging result of the spoken text of the training samples, and the sample tone word tagging result of the spoken text of the training samples.

[0173] The first sample determination module 820 is used to determine the target part of speech of the sample from the part-of-speech tagging results of the first sample based on the distribution characteristics of the sample modal particles.

[0174] The second sample determination module 830 is used to determine the sample prediction insertion position based on the position of the sample content corresponding to the sample target part of speech in the standard text of the training samples.

[0175] The sample masking module 840 is used to mask the sample prediction insertion position in the training sample standard text to obtain the training sample mask standard text, wherein the training sample mask standard text has the first sample part-of-speech tagging result.

[0176] The training module 850 is used to train the initial speech-text generation model using the target training set to obtain the trained speech-text generation model. The target training set includes the training sample mask standard text, the second sample part-of-speech tagging results of the training sample spoken text, and the sample tone word tagging results of the training sample spoken text.

[0177] According to embodiments of this disclosure, the training apparatus for the speech-text generation model may further include: a sample update module and a sample construction module.

[0178] The sample update module is used to update the first sample standard text and the first sample spoken text associated with the first sample standard text in the first sample set using the sample confusion words in the sample confusion dictionary, so as to obtain a second sample set containing the second sample standard text and the second sample spoken text.

[0179] The sample construction module is used to construct a training sample set based on the first sample set and the second sample set.

[0180] According to embodiments of this disclosure, the training apparatus for the speech-text generation model may further include: a corpus processing module, a recognition module, and a confusion dictionary construction module.

[0181] The corpus processing module is used to process the sample standard corpus text using a speech synthesis device to obtain sample speech corpus.

[0182] The recognition module is used to perform speech recognition on the sample speech corpus to obtain the sample confused speech corpus text.

[0183] The confusion dictionary building module is used to build a sample confusion dictionary based on the sample standard corpus text and the sample confusion corpus text.

[0184] According to embodiments of this disclosure, the training apparatus for the speech-text generation model may further include: a sample initial corpus acquisition module, a sample corpus annotation module, and a sample third determination module.

[0185] The initial sample corpus acquisition module is used to acquire the initial sample corpus set, which includes the initial sample spoken corpus text generated based on the sample spoken speech corpus;

[0186] The sample corpus annotation module is used to perform part-of-speech tagging on the initial spoken language corpus text, and obtain the part-of-speech tagging results and the tone word tagging results of the sample spoken language corpus.

[0187] The third sample determination module is used to determine the distribution characteristics of sample modal particles based on the part-of-speech tagging results and the modal particle tagging results of the sample spoken language corpus.

[0188] It should be noted that the training device part of the speech-text generation model in the embodiments of this disclosure corresponds to the training method part of the speech-text generation model in the embodiments of this disclosure. For a detailed description of the training device part of the speech-text generation model, please refer to the training method part of the speech-text generation model, which will not be repeated here.

[0189] Any one or more of the modules or units according to the embodiments of this disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of this disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules or units according to the embodiments of this disclosure can be at least partially implemented as hardware circuitry, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a System-on-Chip, a System-on-a-Substrate, a System-on-Package, an Application-Specific Integrated Circuit (ASIC), or implemented by hardware or firmware in any other reasonable manner by integrating or packaging the circuitry, or implemented in any one of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, one or more of the modules or units according to the embodiments of this disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0190] For example, any plurality of the annotation module 710, the first determining module 720, the second determining module 730, the insertion module 740, and the generation module 750 can be combined into one module / unit, or any one of these modules / units can be split into multiple modules / units. Alternatively, at least part of the functionality of one or more of these modules / units can be combined with at least part of the functionality of other modules / units / subunits and implemented in one module / unit. According to embodiments of the present disclosure, at least one of the annotation module 710, the first determining module 720, the second determining module 730, the insertion module 740, and the generation module 750 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the annotation module 710, the first determining module 720, the second determining module 730, the insertion module 740, and the generation module 750 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0191] Figure 9 A block diagram of an electronic device suitable for implementing a speech-to-text generation method and a training method for a speech-to-text generation model, according to embodiments of the present disclosure, is shown schematically. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0192] like Figure 9 As shown, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0193] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 902 and / or RAM 903. It should be noted that the programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0194] According to embodiments of this disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The system 900 may also include one or more of the following components connected to the I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0195] According to embodiments of this disclosure, the method flow according to embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by processor 901, it performs the functions defined in the system of embodiments of this disclosure. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0196] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0197] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0198] For example, according to embodiments of this disclosure, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0199] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the methods provided in the embodiments of this disclosure.

[0200] When the computer program is executed by the processor 901, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0201] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0202] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0203] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not expressly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0204] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for generating speech-to-text, comprising: Perform part-of-speech tagging on the standard text to obtain the part-of-speech tagging results; The target part of speech is determined from the part-of-speech tagging results based on the distribution characteristics of modal particles, wherein the target part of speech includes verbs and adjectives; The predicted insertion position is determined based on the position of the content corresponding to the target part of speech in the standard text; The target modal particle is inserted into the standard text according to the predicted insertion position to obtain the target spoken text; and Generate target speech text based on the target spoken text; The method further includes: Obtain an initial corpus set, wherein the initial corpus set includes initial spoken language text generated based on spoken speech corpus; The initial spoken language corpus text is tagged with parts of speech to obtain the spoken language corpus part of speech tagging results and the spoken language corpus modal particle tagging results; Based on the part-of-speech tagging results and the modal particle tagging results of the spoken language corpus, the distribution features of the modal particles are determined, and the distribution features of the modal particles represent at least one of the following: Statistical probability of the position of modal particles in spoken language corpus following words with verb function; The statistical probability of the position of the spoken language corpus modal particles before the spoken language corpus words with adjective function, wherein the initial spoken language corpus text includes the spoken language words and the spoken language modal particles.

2. The speech-to-text generation method according to claim 1, wherein, Based on the predicted insertion position, the target modal particle is inserted into the standard text to obtain the target spoken text, including: The predicted insertion position of the standard text is masked to obtain the masked standard text; The masked standard text is input into the speech text generation model so that the speech text generation model inserts the target modifier at the target insertion position in the predicted insertion position, thereby generating the target spoken text.

3. The speech-to-text generation method according to claim 1, wherein part-of-speech tagging is performed on the standard text to obtain the part-of-speech tagging results includes: The standard text is input into the semantic recognition model to obtain the part-of-speech tagging results; The semantic recognition model includes: The first semantic recognition model is constructed based on a recurrent neural network model and a conditional random field model; or A second semantic recognition model based on dependency parsing.

4. A training method for a speech-to-text generation model, comprising: Part-of-speech tagging is performed on the standard training text and the spoken training text associated with the standard training text in the training sample set, respectively, to obtain the first part-of-speech tagging result of the standard training text, the second part-of-speech tagging result of the spoken training text, and the sample modal word tagging result of the spoken training text. The target part of speech of the sample is determined from the part-of-speech tagging results of the first sample based on the distribution characteristics of the sample modal particles. The target part of speech of the sample includes verbs and adjectives. The sample prediction insertion position is determined based on the position of the sample content corresponding to the target part of speech of the sample in the standard text of the training sample. The sample prediction insertion positions in the training sample standard text are masked to obtain the training sample mask standard text, wherein the training sample mask standard text has the first sample part-of-speech tagging result; An initial speech-text generation model is trained using a target training set to obtain a trained speech-text generation model. The target training set includes the training sample mask standard text, the second sample part-of-speech tagging results of the training sample spoken text, and the sample tone word tagging results of the training sample spoken text. The method further includes: Obtain an initial sample corpus set, wherein the initial sample corpus set includes initial sample spoken corpus text generated based on sample spoken speech corpus; The initial spoken language corpus text of the sample is tagged with parts of speech to obtain the part-of-speech tagging results and the tone mark results of the sample spoken language corpus; Based on the part-of-speech tagging results and the modal particle tagging results of the sample spoken language corpus, the distribution features of the sample modal particles are determined, and the distribution features of the sample modal particles represent at least one of the following: The statistical probability of the position of modal particles in the sample spoken language corpus following words in the sample spoken language corpus with verb parts of speech; The statistical probability of the position of modal particles in the sample spoken language corpus preceding words in the spoken language corpus that have the function of adjectives, wherein the initial spoken language corpus text includes the words in the sample spoken language corpus and the modal particles in the sample spoken language corpus.

5. The training method according to claim 4 further includes: The first sample standard text and the first sample spoken text associated with the first sample standard text in the first sample set are updated respectively using the sample confusion words in the sample confusion dictionary to obtain a second sample set containing the second sample standard text and the second sample spoken text. The training sample set is constructed based on the first sample set and the second sample set.

6. The training method according to claim 5 further includes: The standard sample text was processed using a speech synthesis device to obtain the sample speech data; Speech recognition is performed on the sample speech corpus to obtain sample confused speech corpus text; The sample confusion dictionary is constructed based on the sample standard corpus text and the sample confusion corpus text.

7. A speech-to-text generation device, comprising: The annotation module is used to perform part-of-speech tagging on standard text and obtain the part-of-speech tagging results. The first determining module is used to determine the target part of speech from the part-of-speech tagging results based on the distribution characteristics of modal particles, wherein the target part of speech includes verbs and adjectives; The second determining module is used to determine the predicted insertion position based on the position of the content corresponding to the target part of speech in the standard text; An insertion module is used to insert a target modal particle into the standard text according to the predicted insertion position, thereby obtaining the target spoken text; and The generation module is used to generate target speech text based on the target spoken text; The device further includes: An acquisition module is used to acquire an initial corpus set, wherein the initial corpus set includes initial spoken corpus text generated based on spoken speech corpus; The corpus annotation module is used to perform part-of-speech tagging on the initial spoken language corpus text to obtain spoken language corpus part-of-speech tagging results and spoken language corpus modal particle tagging results; The third determining module is used to determine the distribution features of the modal particles based on the part-of-speech tagging results and the modal particle tagging results of the spoken language corpus, wherein the distribution features of the modal particles represent at least one of the following: Statistical probability of the position of modal particles in spoken language corpus following words with verb function; The statistical probability of the position of the spoken language corpus modal particles before the spoken language corpus words with adjective function, wherein the initial spoken language corpus text includes the spoken language words and the spoken language modal particles.

8. A training device for a speech-to-text generation model, comprising: The sample annotation module is used to perform part-of-speech tagging on the standard training sample text and the spoken training sample text associated with the standard training sample text in the training sample set, respectively, to obtain the first part-of-speech tagging result of the standard training sample text, the second part-of-speech tagging result of the spoken training sample text, and the sample tone word tagging result of the spoken training sample text. The first sample determination module is used to determine the target part of speech of the sample from the first sample part-of-speech tagging results based on the distribution characteristics of the sample modal particles. The target part of speech of the sample includes verbs and adjectives. The second sample determination module is used to determine the sample prediction insertion position based on the position of the sample content corresponding to the target part of speech of the sample in the standard text of the training sample; The sample masking module is used to mask the sample prediction insertion position in the training sample standard text to obtain the training sample mask standard text, wherein the training sample mask standard text has the first sample part-of-speech tagging result; The training module is used to train an initial speech-text generation model using a target training set to obtain a trained speech-text generation model. The target training set includes the training sample mask standard text, the second sample part-of-speech tagging results of the training sample spoken text, and the sample tone word tagging results of the training sample spoken text. The training device further includes: The initial sample corpus acquisition module is used to acquire an initial sample corpus set, wherein the initial sample corpus set includes initial sample spoken corpus text generated based on sample spoken speech corpus; The sample corpus annotation module is used to perform part-of-speech tagging on the initial spoken corpus text of the sample, and obtain the part-of-speech tagging results and the tone word tagging results of the sample spoken corpus; The third sample determination module is used to determine the distribution features of the sample speech corpus based on the part-of-speech tagging results and the modal particle tagging results of the sample speech corpus. The distribution features of the sample modal particles represent at least one of the following: The statistical probability of the position of modal particles in the sample spoken language corpus following words in the sample spoken language corpus with verb parts of speech; The statistical probability of the position of modal particles in the sample spoken language corpus preceding words in the spoken language corpus that have the function of adjectives, wherein the initial spoken language corpus text includes the words in the sample spoken language corpus and the modal particles in the sample spoken language corpus.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 6.

10. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 6.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images