Speech recognition method, conference speech recognition method and speech translation method

Abstract

The embodiment of the invention provides a voice recognition method, a conference voice recognition method and a voice translation method. The voice recognition method comprises the following steps: acquiring an initial recognition text and a voice associated text of a target voice; the initial recognition text and the voice association text are input into a text generation model, a target recognition text of the target voice is obtained, the text generation model is obtained based on training of a sample text pair and a sample association text carried by the sample text pair, and the sample text pair comprises an initial sample text and a target sample text; the target sample text is a written style text. The target sample text is utilized to enable the model to have the text written style rewriting capability, the voice associated text is added in the model input, the understanding of the model on the initial recognition text is deepened, written rewriting of the initial recognition text is realized, and the readability of the target recognition text is enhanced.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular to a speech recognition method, a conference speech recognition method, and a speech translation method. Background Art

[0002] With the development of computer technology, automatic speech recognition (ASR) has achieved good recognition results in many scenarios. Automatic speech recognition enables machines to recognize and understand human voice signals and convert them into text. It belongs to the interdisciplinary field of computer science, computational linguistics, etc.

[0003] At present, automatic speech recognition technology is usually used to transcribe audio and video word by word, so that users can easily obtain the corresponding transcribed text of the audio and video. However, since audio and video usually have spoken style expressions, such as repetition, redundancy, fragmentation, etc., the text obtained by word-by-word transcription is poorly readable. Therefore, a speech processing solution with higher readability is urgently needed. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a speech recognition method. One or more embodiments of this specification also relate to a conference speech recognition method, a speech translation method, a text generation model training method, an information processing method based on a text generation model, a task platform, a speech recognition device, a conference speech recognition device, a speech translation device, a text generation model training device, an information processing device based on a text generation model, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, there is provided a speech recognition method, comprising:

[0006] Obtaining the initial recognition text and speech-related text of the target speech;

[0007] The initial recognition text and the speech-associated text are input into a text generation model to obtain a target recognition text of the target speech, wherein the text generation model is trained based on a sample text pair and a sample-associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style.

[0008] According to a second aspect of the embodiments of this specification, a conference speech recognition method is provided, including:

[0009] Obtaining the initial recognition text and speech-related text of the target conference speech;

[0010] The initial recognition text and speech-associated text are input into a text generation model to obtain a target recognition text of the target conference speech, wherein the text generation model is trained based on sample text pairs and sample-associated texts carried by the sample text pairs, the sample text pairs include initial sample text and target sample text, and the target sample text is a written-style text.

[0011] According to a third aspect of the embodiments of this specification, a speech translation method is provided, including:

[0012] Obtaining a target recognition text of a target speech, wherein the target recognition text is generated by a text generation model based on an initial recognition text of the target speech and speech-related text, and the text generation model is trained based on a sample text pair and a sample-related text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style;

[0013] Translate the target recognized text to obtain the text translation result of the target speech.

[0014] According to a fourth aspect of the embodiments of this specification, a text generation model training method is provided, including:

[0015] Obtaining a sample text pair, wherein the sample text pair carries a sample associated text, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style;

[0016] Inputting the initial sample text and the sample associated text into the initial generation model to obtain the sample predicted text;

[0017] According to the sample prediction text and the target sample text, the model parameters of the initial generation model are adjusted to obtain a trained text generation model.

[0018] According to a fifth aspect of the embodiments of this specification, there is provided an information processing method based on a text generation model, comprising:

[0019] receiving a task generation request, wherein the task generation request includes request information;

[0020] Based on the request information, a text generation model is obtained, wherein the text generation model is trained based on a sample text pair and a sample associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style;

[0021] Based on the text generation model, task information is generated, wherein the task information is used to perform a target speech task.

[0022] According to a sixth aspect of the embodiments of this specification, there is provided a task platform, including a request interface and a response unit;

[0023] A request interface, used to receive a task generation request, wherein the task generation request includes request information;

[0024] The response unit is used to obtain a text generation model based on the request information, wherein the text generation model is trained based on a sample text pair and a sample associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a written style text; based on the text generation model, task information is generated, wherein the task information is used to perform a target speech task.

[0025] According to a seventh aspect of the embodiments of this specification, a speech recognition device is provided, including:

[0026] A first acquisition module is configured to acquire an initial recognition text and a speech-related text of a target speech;

[0027] The first input module is configured to input the initial recognition text and the speech-associated text into the text generation model to obtain the target recognition text of the target speech, wherein the text generation model is trained based on the sample text pairs and the sample-associated texts carried by the sample text pairs, the sample text pairs include the initial sample text and the target sample text, and the target sample text is a written-style text.

[0028] According to an eighth aspect of the embodiments of this specification, a conference speech recognition device is provided, including:

[0029] A second acquisition module is configured to acquire an initial recognition text and a speech-related text of the target conference speech;

[0030] The second input module is configured to input the initial recognition text and the speech-associated text into the text generation model to obtain the target recognition text of the target conference speech, wherein the text generation model is trained based on the sample text pairs and the sample-associated texts carried by the sample text pairs, the sample text pairs include the initial sample text and the target sample text, and the target sample text is a written-style text.

[0031] According to a ninth aspect of the embodiments of this specification, a speech translation device is provided, including:

[0032] A third acquisition module is configured to acquire a target recognition text of a target speech, wherein the target recognition text is generated by a text generation model based on an initial recognition text of the target speech and speech-associated text, and the text generation model is trained based on a sample text pair and a sample-associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style;

[0033] The translation module is configured to translate the target recognition text to obtain a text translation result of the target speech.

[0034] According to a tenth aspect of the embodiments of this specification, a text generation model training device is provided, including:

[0035] A fourth acquisition module is configured to acquire a sample text pair, wherein the sample text pair carries a sample associated text, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style;

[0036] A third input module is configured to input the initial sample text and the sample associated text into the initial generation model to obtain the sample predicted text;

[0037] The adjustment module is configured to adjust the model parameters of the initial generation model according to the sample prediction text and the target sample text to obtain a trained text generation model.

[0038] According to an eleventh aspect of the embodiments of this specification, there is provided an information processing device based on a text generation model, comprising:

[0039] A receiving module is configured to receive a task generation request, wherein the task generation request includes request information;

[0040] A fifth acquisition module is configured to acquire a text generation model based on the request information, wherein the text generation model is trained based on a sample text pair and a sample associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style;

[0041] The generation module is configured to generate task information based on the text generation model, wherein the task information is used to perform the target speech task.

[0042] According to a twelfth aspect of the embodiments of this specification, a computing device is provided, including:

[0043] Memory and processor;

[0044] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method provided in the first aspect, the second aspect, the third aspect, the fourth aspect, or the fifth aspect are implemented.

[0045] According to the thirteenth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the method provided in the first aspect, the second aspect, the third aspect, the fourth aspect, or the fifth aspect are implemented.

[0046] According to the fourteenth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the steps of the method provided in the first aspect, the second aspect, the third aspect, the fourth aspect, or the fifth aspect.

[0047] The speech recognition method provided by an embodiment of the present specification includes: obtaining the initial recognition text and speech-related text of the target speech; inputting the initial recognition text and speech-related text into a text generation model to obtain the target recognition text of the target speech, wherein the text generation model is trained based on a sample text pair and a sample-related text carried by the sample text pair, and the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style. Since the target sample text is a text in a written style, the text generation model is obtained by training with the target sample text, so that the text generation model has the ability to rewrite the text in a written style, and adding speech-related text to the input of the text generation model deepens the understanding of the initial recognition text by the text generation model, ensuring that the target recognition text can fully and accurately reflect the information in the target speech, and at the same time realizes the written rewriting of the initial recognition text, thereby enhancing the readability of the target recognition text. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of a speech recognition method provided by an embodiment of this specification;

[0049] Figure 2 is a schematic diagram of a process for generating a sample text pair provided by an embodiment of this specification;

[0050] Figure 3 is a flow chart of a conference speech recognition method provided by an embodiment of this specification;

[0051] Figure 4 is a flow chart of a speech translation method provided by an embodiment of this specification;

[0052] Figure 5 is a flowchart of a text generation model training method provided by an embodiment of this specification;

[0053] Figure 6 is a flowchart of an information processing method based on a text generation model provided by an embodiment of this specification;

[0054] Figure 7 It is a structural diagram of a task platform provided by an embodiment of this specification;

[0055] Figure 8 is a structural diagram of a speech recognition device provided by an embodiment of this specification;

[0056] Fig. 9 It is a structural diagram of a conference speech recognition device provided by an embodiment of this specification;

[0057] Fig.10 It is a structural schematic diagram of a speech translation device provided by an embodiment of this specification;

[0058] Fig.11 It is a structural schematic diagram of a text generation model training device provided by an embodiment of this specification;

[0059] Fig.12 It is a structural schematic diagram of an information processing device based on a text generation model provided by an embodiment of this specification;

[0060] Fig.13 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0061] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0062] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0063] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0064] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0065] In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, which usually contains hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than 10 trillion model parameters. A large model can also be called a foundation model / foundation model. The large model is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as a large-scale language model (LLM, Large Language Model), a multi-modal pre-training model, etc.

[0066] When the big model is used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. The big model can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image description (IC, Image Caption), image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of the big model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

[0067] First, the terms involved in one or more embodiments of this specification are explained.

[0068] Oral text writtenization: It refers to rewriting text in an oral style into text in a written style while comprehensively and accurately retaining the information in the original content.

[0069] Inverse Text Normalization (ITN): It is the conversion of non-standard forms in oral expressions in natural language into standard and structured forms for purposes such as computer processing, data storage, or information retrieval. In the Chinese processing scenario, ITN technology can be used to process the normalization of time, dates, numbers, and other specific expressions. For example, converting "三.1" to "3.1" and "三十号晚上" to the standardized format of "the evening of May 30, 2024", aiming to ensure that the system can accurately understand and generate standardized information that meets expectations.

[0070] Grammar Error Correction (GEC): It is a natural language processing technology that focuses on automatically detecting and correcting grammar errors in text and is widely used in educational software, writing assistance tools, post-editing of machine translation, and any scenario that requires improving the quality of text. The GEC system can identify various grammar problems in a sentence, such as spelling mistakes, punctuation mistakes, tense mistakes, subject-verb disagreement, word order mistakes, etc., and provide corresponding modification suggestions to help users or the system output more accurate and fluent language expressions.

[0071] Disfluency Removal: It refers to removing elements that interfere with the normal fluency of oral expressions from oral expressions, such as filler words (such as "嗯", "啊"), repeated words, self-corrections, interruptions, and unnecessary filler words, etc., making the final text or speech clearer, more fluent, and easier to understand.

[0072] BLEU (Bilingual Evaluation Understudy): It is an automatic evaluation metric used to evaluate the quality of text based on the degree of n-gram matching, taking into account factors such as precision and brevity penalty.

[0073] ROUGE (Recall-Oriented Understudy for Gisting Evaluation): It is used to measure the content coverage between the text generated by the system and the reference text based on recall rate.

[0074] With the development of ASR technology, users can obtain the corresponding transcribed text of audio and video more and more conveniently. ASR is committed to transcribing speech into corresponding text as accurately as possible. This kind of transcription is like an echo of what is said. It is a verbatim transcription of spoken language. It is often disorganized and contains many oral expression features such as modal particles, repetition, redundancy, and fragmentation. It even contains some recognition errors (especially recognition errors of entity words and professional terms) and grammatical errors, resulting in poor readability of the transcribed text.

[0075] In order to solve the above problems, the embodiments of this specification propose a solution for writing spoken text based on a large model. By adding additional voice-related text to the input of the large model, the large model is used to automatically rewrite the spoken text, correct ASR recognition errors, grammatical errors and other problems, and convert the spoken style expression into an easy-to-read written style expression, thereby converting the "speaking" content into the "reading" content, comprehensively and accurately reflecting the original information and enhancing the readability of the text.

[0076] It is worth noting that the speech recognition method provided in the embodiments of this specification is generally performed by the server, but in other embodiments of this specification, the client may also have similar functions to the server, thereby performing the speech recognition method provided in the embodiments of this specification. In other embodiments, the speech recognition method provided in the embodiments of this specification may also be performed jointly by the client and the server.

[0077] In this specification, a speech recognition method is provided. This specification also involves a conference speech recognition method, a speech translation method, a text generation model training method, an information processing method based on a text generation model, a task platform, a speech recognition device, a conference speech recognition device, a speech translation device, a text generation model training device, an information processing device based on a text generation model, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0078] See also Figure 1 , Figure 1 A flow chart of a speech recognition method provided by an embodiment of the present specification is shown, which specifically includes the following steps:

[0079] Step 102: Obtain the initial recognition text and speech-associated text of the target speech.

[0080] Exemplarily, the target voice refers to the sound containing spoken content in the audio and video. The target voice can come from different scenes, such as conference scenes, podcast scenes, speech scenes, chat scenes, interview scenes, interview scenes, open class scenes, etc. The spoken content included in the target voice can be content in different languages, such as the target voice can be Chinese voice, English voice, etc. The initial recognition text of the target voice refers to the recognition text obtained by performing ASR on the target voice. Since the recognition text obtained by ASR is a word-for-word transcription of the spoken expression, the initial recognition text can be understood as a spoken style expression text.

[0081] Speech-related text refers to text that is associated with the target speech, and can be regarded as context or auxiliary information for the target speech extension. Exemplarily, assuming that the target speech is a speech in a podcast scene, the speech-related text can be the guest name of the podcast, the podcast professional terminology, the podcast discussion topic, etc. Assuming that the target speech is a speech in a meeting scene, the speech-related text can be the meeting title, meeting progress, meeting decision, meeting summary, meeting agenda, etc. Assuming that the target speech is a speech in a speech scene, the speech-related text can be the speaker information (such as personal profile), speech title, speech category, etc. It should be noted that the speech-related text can be used as a reference in the text generation model for text rewriting, which helps to eliminate challenging pronunciations in the initial recognition text, especially some names, terms in specific fields or named entities, etc. In addition, speech-related text also provides more background knowledge, topic semantics, and habitual expressions, which will also help the written version of spoken language.

[0082] In actual applications, there are many ways to obtain the initial recognition text and voice-associated text of the target speech, which can be selected according to the actual situation, and the embodiments of this specification do not impose any restrictions on this. For the initial recognition text, in one possible implementation, the initial recognition text of the target speech can be read from other data acquisition devices or databases. In another possible implementation, the target speech can be processed by ASR to obtain the initial recognition text. For the voice-associated text, in one possible implementation, the introduction information of the target speech can be collected as voice-associated text, such as using the podcast introduction as the voice-associated text of the speech in the podcast scene. In another possible implementation, the voice-associated text of the target speech sent by the client can be received.

[0083] Step 104: Input the initial recognition text and the speech-associated text into a text generation model to obtain a target recognition text for the target speech, wherein the text generation model is trained based on a sample text pair and a sample-associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a written-style text.

[0084] Exemplarily, the text generation model is used to rewrite the initial recognition text based on the speech-associated text to obtain the target recognition text. The purpose of text rewriting is to correct ASR errors and grammatical errors in the initial recognition text, and to eliminate the spoken expression features in the initial recognition text, so as to obtain the target recognition text expressed in a written style while faithfully retaining the original content. The target sample text is obtained based on the written style processing of the original predicted text of the original speech, and the original predicted text is generated by the initial generation model based on the original recognition text and the original associated text of the original speech. The text generation model can be a pre-trained large model trained using the zero shot capability of a large model, or it can be a deep learning model trained based on sample text pairs and sample associated texts carried by the sample text pairs.

[0085] For example, assume that the speech-associated text is "Discussing Zhang's movie Happy with KerryLee and GillWang...". The initial recognition text is "Hello Ms.Carrie.I am Jill.What topic are we discussing today?Everyone seeing the title,you already know.A blockbuster movie.Zhang Happy.Yes,aha,Happy,it feels.This movie has achieved very high box office.". The initial recognition text and the speech-associated text are input into the text generation model, and the target recognition text of the target speech is obtained as "Hello Ms.Kerry.I'm Gill.As everyone can tell from the title,today wediscuss Zhang'sblockbuster movie—"Happy",which has achieved very high box office revenue.". In the target recognition text, the ASR errors “Carrie” and “Jill” were rewritten into the correct “Kerry” and “Gill” respectively, the grammatical error “has achieve” was rewritten into the correct “has achieved”, and the spoken style statement “What topic are we discussing today? Everyone seeing the title, you already know. A blockbuster movie. Zhang Happy. Yes, aha, Happy, it feels.” was rewritten into the written style statement “As everyone can tell from the title, today we discuss Zhang's blockbuster movie—"Happy"”.

[0086] By applying the scheme of the embodiments of this specification, since the target sample text is a written-style text, a text generation model is obtained by training with the target sample text, so that the text generation model has the ability to rewrite the text in a written style. Adding speech-related text to the input of the text generation model deepens the text generation model's understanding of the initial recognition text, ensuring that the target recognition text can comprehensively and accurately reflect the information in the target speech, and at the same time realizes the written rewriting of the initial recognition text, thereby enhancing the readability of the target recognition text.

[0087] In an optional embodiment of the present specification, since there may be multiple errors in the initial recognition text, such as recognition errors, punctuation errors, irregularities, etc., therefore, when using the text generation model to correct the errors in the initial recognition text, additional text generation prompt information can be added to the input of the text generation model, and the text generation prompt information is used to guide the text generation model to perform accurate speech recognition post-processing tasks, that is, the text generation prompt information, the initial recognition text, and the speech-related text are input into the text generation model to obtain the target recognition text of the target speech. In practical applications, the text generation prompt information can be obtained, the text generation prompt information, the initial recognition text, and the speech-related text are fused, and the fused information is input into the text generation model to obtain the target recognition text of the target speech.

[0088] It should be noted that the text generation prompt information may include a speech recognition standard, thereby prompting the text generation model to perform a speech recognition post-processing task to obtain a target recognition text that meets the speech recognition standard and is expressed in a written style. Therefore, the speech recognition standard can also be referred to as a written style processing standard. The speech recognition post-processing task can be understood as a contextualized spoken written conversion (CoS2W, Contextualized Spoken2Written Conversion) task. The speech recognition post-processing task may include at least one of an ASR error correction task, an ITN error correction task, a punctuation error correction task, a GEC error correction task, a DisfluencyRemoval task, and a written style transfer task. Exemplarily, the multiple subtasks included in the speech recognition post-processing task have a coupling relationship. For example, ASR errors and grammatical errors will change the semantics of the sentence to a certain extent, thereby affecting the performance of the written style transfer task. For example, the performance of GEC on written texts is usually better than that on spoken texts, and the way of expressing the spoken style will interfere with the performance of the GEC task. Therefore, each subtask is important for ensuring the readability of the final result. Preferably, the speech recognition post-processing task can be an organic combination of multiple subtasks.

[0089] Exemplarily, the text generation prompt information can be: Assume that you are a professional recorder. Please improve the initial recognition text in a written and formal form. The target recognition text obtained after improvement should be free of obvious ASR errors and grammatical errors, faithfully preserve the original content, and show a clear writing style and good readability. Please refer to the given speech-related text. The initial recognition text is a speech transcription text using ASR, usually in a colloquial or informal style, in the format of [line identifier][transcription]. The relevant speech recognition standards are as follows: 1. Please correct grammar, spelling and ASR errors by referring to the speech-related text. 2. Please convert informal styles such as redundancy and repetition into formal styles. 3. Please improve its fluency and coherence. If appropriate, you can adjust the sentence structure. 4. The improved output must faithfully preserve the information of the given content. Do not add, delete or modify important information. 5. Please keep the original language of multilingual content. 6. The number of lines of the output after improvement is consistent with the given content. The output format is [line identifier][result]. The following is the given speech-associated text [speech-associated text], the following is the initial recognition text [initial recognition text], and the following is the target recognition text [].

[0090] In an optional embodiment of the present specification, since the input length of the text generation model is usually limited, in order to ensure that the text generation model can perform text rewriting normally, the speech-related text can be processed according to the text processing strategy to reduce the length of the speech-related text, that is, before the initial recognition text and the speech-related text are input into the text generation model to obtain the target recognition text of the target speech, the following steps can also be included:

[0091] The speech-related text is updated according to a text processing strategy, wherein the text processing strategy includes at least one of a summary extraction strategy, a keyword extraction strategy, and a text retrieval strategy.

[0092] It should be noted that the text processing strategy is used to analyze, understand, convert, and extract valuable information from speech-related texts, where the valuable information is the updated speech-related texts. The summary extraction strategy is used to extract short and representative summaries from speech-related texts. The summary can retain the main information and key points of the speech-related texts. According to different methods, summaries can be divided into two categories: extractive summaries and generative summaries. Extractive summaries directly select key sentences or phrases from speech-related texts to form summaries; generative summaries generate new sentences as summaries by understanding and reorganizing speech-related texts. The keyword extraction strategy is used to extract keywords from speech-related texts that can summarize their themes and core information. Methods for extracting keywords include methods based on term frequency-inverse document frequency (TF-IDF), methods based on graph theory (such as TextRank), and technologies that combine machine learning and deep learning. The text retrieval strategy is used to retrieve text fragments with a high degree of relevance to the initial recognition text from speech-related texts. Text retrieval methods include text similarity calculation methods, retrieval methods based on machine learning models, and so on.

[0093] In practical applications, according to the text processing strategy, the voice-associated text can be updated to obtain the updated voice-associated text; the above-mentioned inputting the initial recognition text and the voice-associated text into the text generation model to obtain the target recognition text of the target voice can include the following steps: inputting the initial recognition text and the updated voice-associated text into the text generation model to obtain the target recognition text of the target voice.

[0094] By applying the solution of the embodiment of this specification, the speech-related text is processed through a text processing strategy, so that the updated speech-related text is shorter while representing the original content, thereby ensuring the accurate processing of the text generation model.

[0095] Because context plays an important role in text understanding, it is particularly obvious in spoken text. In many spoken interaction scenarios, especially high-frequency interaction scenarios (such as meetings), sentence fragmentation is very common, and the meaning of the sentence can be properly understood by relying on the context. On the other hand, if only a single sentence is rewritten without starting from a larger granularity, the readability of the rewritten text is limited. By modifying multiple sentences with a larger granularity, the repetition and redundancy problems at the sentence level can be alleviated, and the result expression can be better improved. In addition, by adjusting the word order, the expression can be made clearer and the reading experience can be enhanced. Therefore, in the embodiment of this specification, when the initial recognition text is rewritten using the text generation model, the initial recognition text can be divided into multiple initial text blocks, and the text context of each initial text block is determined, and the text context is integrated into the processing process of the text generation model, that is, the initial recognition text includes multiple initial text blocks, and the initial text block includes at least one paragraph in the initial recognition text. The above-mentioned initial recognition text and speech-related text are input into the text generation model, and before the target recognition text of the target speech is obtained, the following steps can also be included:

[0096] Dividing the initial recognized text into paragraphs to obtain a plurality of initial text blocks, wherein the initial text block includes at least one paragraph in the initial recognized text;

[0097] Determine text contexts of a plurality of initial text blocks respectively from the initial recognized text;

[0098] Inputting the initial recognition text and the speech-associated text into the text generation model to obtain the target recognition text of the target speech may include the following steps:

[0099] The speech-associated text, the multiple initial text blocks, and the text contexts of the multiple initial text blocks are input into a text generation model to obtain a target recognition text of the target speech.

[0100] Exemplarily, the text context of the initial text block is extracted from the initial recognition text based on the initial text block. If the initial recognition text is directly input into the text generation model, the process of the text generation model performing text rewriting can be regarded as the text rewriting of the chapter granularity. Since the text rewriting content of the chapter granularity is large, the accuracy of the target recognition text may be low. Therefore, the initial recognition text can also be divided into contents of different granularities, and the text rewriting can be flexibly performed, that is, the initial recognition text is divided into paragraphs to obtain multiple initial text blocks. If the initial text block includes a paragraph, the process of the text generation model performing text rewriting can be regarded as the text rewriting of the paragraph granularity (paragraph-level). If the initial text block includes multiple paragraphs, the process of the text generation model performing text rewriting can be regarded as the text rewriting of the fragment granularity (chunk-level). The text context is a text fragment in the initial recognition text that has a content association relationship with the initial text block.

[0101] It should be noted that there are multiple ways to divide the initial recognized text into paragraphs and obtain multiple initial text blocks, which can be selected based on actual conditions, and the embodiments of this specification do not impose any restrictions on this. In one possible implementation, a preset number of initial text blocks can be obtained, and the initial recognized text can be divided into a preset number of initial text blocks in units of paragraphs. In another possible implementation, a preset length threshold of the initial text block can be obtained, and the initial recognized text can be divided into multiple initial text blocks in units of paragraphs, and the length of each initial text block does not exceed the preset length threshold.

[0102] By applying the solution of the embodiments of this specification, the text context of each initial text block is determined, and the text context is used to assist the processing of the text generation model, thereby improving the performance of the text generation model in context-related spoken-to-written tasks and improving the accuracy of target recognition text.

[0103] In an optional embodiment of the present specification, the above-mentioned inputting the speech-associated text, the multiple initial text blocks, and the text context of the multiple initial text blocks into the text generation model to obtain the target recognition text of the target speech may include the following steps:

[0104] Repeat the following steps until multiple target text blocks corresponding to the initial text blocks are obtained:

[0105] Filtering out a first initial text block from the multiple initial text blocks, wherein the first initial text block is any one of the multiple initial text blocks;

[0106] Inputting the speech-associated text, the first initial text block, and the text context of the first initial text block into a text generation model to obtain a target text block corresponding to the first initial text block;

[0107] The target text blocks corresponding to the multiple initial text blocks are used to construct the target recognition text of the target speech.

[0108] It should be noted that the speech-related text, multiple initial text blocks, and the text contexts of the multiple initial text blocks are input into the text generation model, and the text generation model can perform multiple rounds of text rewriting on the initial text blocks one by one, taking the initial text blocks as units. The target text block of the first initial text block refers to the text block obtained by rewriting the first initial text block based on the text context of the first initial text block and the speech-related text. The target text block corresponding to the first initial text block can realize the conversion from spoken style expression to written style expression under the premise of faithfully retaining the content of the first initial text block.

[0109] Exemplarily, assuming that the initial recognition text of the target speech includes three paragraphs (paragraph 1, paragraph 2, and paragraph 3), the initial recognition text is divided into paragraphs to obtain two initial text blocks, which are initial text block 1 and initial text block 2, respectively, wherein initial text block 1 includes paragraph 1 and paragraph 2, and initial text block 2 includes paragraph 3. From the initial recognition text, the text context of initial text block 1 is determined to be text context 1, and the text context of initial text block 2 is determined to be text context 2. The speech-associated text, initial text block 1, and text context 1 are input into the text generation model to obtain the target text block 1 corresponding to the initial text block 1. The speech-associated text, initial text block 2, and text context 2 are input into the text generation model to obtain the target text block 2 corresponding to the initial text block 2. The target text block 1 and the target text block 2 are concatenated to obtain the target recognition text of the target speech.

[0110] In actual applications, the speech-related text, the first initial text block, and the text context of the first initial text block are input into the text generation model. When the target text block corresponding to the first initial text block is obtained, text generation prompt information can be obtained, and the text generation prompt information, the speech-related text, the first initial text block, and the text context of the first initial text block are fused, and the fused information is input into the text generation model to obtain the target recognition text of the target speech. In addition to the content in the above example, the text generation prompt information can also include "The following is the initial recognition text [first initial text block], and the following is the given text context [text context of the first initial text block]".

[0111] By applying the solution of the embodiments of this specification, since the initial text block includes at least one paragraph, text rewriting is performed based on the initial text block, which can alleviate sentence-level repetition and redundancy problems and better improve the readability of the target recognition text.

[0112] In an optional embodiment of the present specification, the above-mentioned determining the text contexts of the multiple initial text blocks from the initial recognition text may include the following steps:

[0113] According to the positions of the multiple initial text blocks in the initial recognized text and / or the content association relationship between the multiple initial text blocks and the initial recognized text, the text contexts of the multiple initial text blocks are respectively determined from the initial recognized text.

[0114] It should be noted that the text context includes at least one of a local context and a global context. The local context refers to a paragraph in the initial recognition text that is adjacent to the initial text block in position. The global context refers to a paragraph in the initial recognition text that is not adjacent to the initial text block in position but is semantically related. Since the initial text block includes at least one paragraph, the initial recognition text naturally includes the local context of the initial text block. For the global context, the content association relationship between the initial text block and the initial recognition text can be determined by calculating the cosine similarity, or the content association relationship between the initial text block and the initial recognition text can be determined by an information retrieval method (such as BM25), and then the text context of the initial text block is determined from the initial recognition text based on the content association relationship.

[0115] By applying the solution of the embodiments of this specification, the text context of the initial text block is determined by the position of the initial text block in the initial recognized text and / or the content association relationship between the initial text block and the initial recognized text, so that the text context is more accurate.

[0116] In an optional embodiment of the present specification, after the initial recognition text and the speech-associated text are input into the text generation model to obtain the target recognition text of the target speech, the following steps may also be included:

[0117] Send the target identification text to the client;

[0118] Receive result feedback information sent by the client for the target recognition text;

[0119] Based on the feedback information from the results, build a model to optimize the data;

[0120] Use model optimization data to adjust parameters of the text generation model.

[0121] Exemplarily, the result feedback information may be information that provides feedback on the content, quality, and completion of the target recognition text. The result feedback information reflects the client's true feelings and expectations about the target recognition text. The result feedback information includes, but is not limited to, quality evaluation information, corrected and accurate target recognition text, and optimization areas of the model. Model optimization data refers to accurate optimized sample data used to optimize the text generation model.

[0122] In actual applications, after receiving the result feedback information sent by the client for the target recognition text, optimization prompt information can be generated based on the result feedback information, and model optimization data sent by the client based on the optimization prompt information can be received. When constructing model optimization data based on the result feedback information, if the result feedback information is the corrected and accurate target recognition text, the initial recognition text of the target speech, the speech-related text, and the corrected and accurate target recognition text can be determined as model optimization data. If the result feedback information is the optimization field of the model, multiple sample text pairs in the optimization field can be obtained, and the sample text pairs can be determined as model optimization data.

[0123] By applying the solution of the embodiments of this specification, the performance of the text generation model is continuously optimized by collecting and utilizing result feedback information to more accurately meet the actual needs of the client and improve the quality and accuracy of the final target recognition text.

[0124] In an optional embodiment of the present specification, a training method for a text generation model is described. That is, before the initial recognition text and the speech-associated text are input into the text generation model to obtain the target recognition text of the target speech, the following steps may also be included:

[0125] Obtaining a sample text pair, wherein the sample text pair carries a sample associated text;

[0126] Inputting the initial sample text and the sample associated text in the sample text pair into the initial generation model to obtain the sample predicted text;

[0127] According to the sample prediction text and the target sample text in the sample text pair, the model parameters of the initial generation model are adjusted to obtain a trained text generation model.

[0128] Exemplarily, the training method of the initial generation model is supervised training, that is, the initial sample text carries the target sample text of the written style expression. The target sample text is the generation target of the initial generation model. The sample-related text refers to the text that has content association with the initial sample text, for example, the sample-related text is a question that explains the entity in the initial sample text, and the sample-related text is a summary text of the initial sample text.

[0129] It should be noted that there are multiple ways to obtain sample text pairs, which can be selected based on actual conditions, and the embodiments of this specification do not impose any restrictions on this. In one possible implementation, multiple sample text pairs can be read from a database. In another possible implementation, sample text pairs can be constructed based on the original recognition text and the original associated text of the original speech. The implementation method of "inputting the initial sample text and sample associated text in the sample text pair into the initial generation model to obtain the sample predicted text" is the same as the above-mentioned implementation method of "inputting the initial recognition text and the speech-associated text into the text generation model to obtain the target recognition text of the target speech", and the embodiments of this specification will not be repeated.

[0130] In practical applications, when adjusting the model parameters of the initial generation model according to the sample prediction text and the target sample text in the sample text pair, the loss value can be calculated according to the sample prediction text and the target sample text, and the model parameters of the initial generation model can be adjusted according to the loss value until the training process meets the preset stop condition, and the trained text generation model is obtained. Among them, there are many functions for calculating the loss value, such as the cross entropy loss function, the L1 norm loss function, the maximum loss function, the mean square error loss function, the logarithmic loss function, etc., which are selected according to the actual situation, and the embodiments of this specification do not make any restrictions on this. The preset stop conditions include but are not limited to the loss value being less than or equal to the preset threshold value and the number of iterations reaching the preset number of iterations. The preset threshold value and the preset number of iterations are selected according to the actual situation, and the embodiments of this specification do not make any restrictions on this.

[0131] In one possible implementation of the present specification, after calculating the loss value, the loss value can be compared with a preset threshold. Exemplarily, if the loss value is greater than the preset threshold, it means that the difference between the sample prediction text and the target sample text is large, and the text rewriting ability of the initial generation model is poor. At this time, the model parameters of the initial generation model can be adjusted, and the initial generation model can continue to be trained using the sample text until the loss value is less than or equal to the preset threshold, indicating that the difference between the sample prediction text and the target sample text is small, and the preset stop condition is reached to obtain a text generation model that has been trained. In another possible implementation of the present specification, in addition to comparing the size relationship between the loss value and the preset threshold, it is also possible to determine whether the current initial generation model has been trained in combination with the number of iterations. Exemplarily, if the loss value is greater than the preset threshold, the model parameters of the initial generation model are adjusted, and the initial generation model is continued to be trained using the sample text until the preset number of iterations is reached, and the iteration is stopped to obtain a text generation model that has been trained.

[0132] By applying the scheme of the embodiments of this specification, since the target sample text is obtained by adjusting the original predicted text of the original speech based on the speech recognition standard, a text generation model is obtained by training with the target sample text, so that the text generation model has the ability to generate text that meets the speech recognition standard, thereby ensuring that the target recognition text of the target speech can comprehensively and accurately reflect the information in the target speech.

[0133] See also Figure 2 , Figure 2 A schematic diagram of a sample text pair generation process provided by an embodiment of the present specification is shown. The sample text pair generation process can be divided into three stages: model generation, manual adjustment, and quality inspection and acceptance. Next, these three stages are described respectively.

[0134] Model generation: Get the original speech, perform ASR recognition and divide the original speech into paragraphs to get the original recognized text. According to the paragraphs in the original recognized text and the input length threshold of the text generation model, divide the original recognized text into different segments as the input of the text generation model, and use an end-to-end approach to use the text generation model to complete segment-level ASR error correction, GEC, text rewriting and other tasks. At the same time, the text context of the segment and the original related text are additionally added to the input of the text generation model as references to improve the text generation model's understanding of the content of the original recognized text and enhance the performance of ASR error correction. At the same time, during the model generation process, the correspondence between paragraphs can be achieved to keep the model input segment and output segment consistent in paragraph granularity;

[0135] Manual adjustment: For the original prediction text generated by the text generation model, there may be problems such as serialization, fidelity errors, and insufficient written text. Therefore, it is possible to determine whether the original prediction text has the above problems and whether it is a written style text. In the case where the original prediction text does not have the above problems and is a written style text, the original prediction text can be determined as the original adjustment text; in the case where the original prediction text has any of the above problems or is not a written style text, the original prediction text can be manually adjusted to obtain the original adjustment text to ensure that the original adjustment text does not have the above problems and is a written style text. During manual adjustment, the paragraphs with problems or spoken style in the original prediction text can be automatically screened out, and only the screened paragraphs can be manually adjusted, or the paragraphs with problems or spoken style in the original prediction text can be manually screened out, and the screened paragraphs can be adjusted. Exemplarily, the original recognition text, the original prediction text, and the original associated text can be provided as references, and the original prediction text obtained by model polishing can be manually adjusted to ensure paragraph alignment, and error correction can be performed to remove ASR recognition errors and grammatical errors that still exist in the original prediction text. You can also polish the original predicted text to adjust the content fidelity and writtenness issues, so that the original predicted text content is faithful to the original recognized text and the expression is written and fluent;

[0136] Quality inspection and acceptance: After obtaining the original adjusted text, the quality inspector can review and accept the original adjusted text. For the paragraphs or chapters in the original adjusted text that do not meet the acceptance requirements, they are manually readjusted until the original adjusted text meets the acceptance requirements; for the original adjusted text that meets the acceptance requirements, the original recognition text is determined as the initial sample text, and the original adjusted text is determined as the target sample text to obtain a sample text pair. Among them, the number of original adjusted texts obtained by manual adjustment can be one or more, and the acceptance requirements include but are not limited to fluency requirements, written requirements, and faithfulness requirements, which are not limited in any way in the embodiments of this specification.

[0137] Furthermore, after obtaining the sample text pairs, the sample text pairs can be used to perform supervised fine-tuning (SFT) based on the initial generation model to improve the instruction-following performance of the task.

[0138] In actual applications, the speech recognition method proposed in the embodiments of this specification has been subjected to multiple experiments, such as spoken and written performance experiments of different text generation models, spoken and written performance experiments of different model input granularities, spoken and written performance experiments of different text contexts, spoken and written performance experiments of different speech-associated texts, spoken and written performance experiments of different languages ​​and different scenarios, and evaluation experiments of large models. The above experiments are explained below.

[0139] Experiment on the oral-written performance of different text generation models: When comparing the performance of different text generation models in speech recognition tasks, in order to test the ability of the text generation model to use context and reduce the consumption of tokens, the model baseline can be set to chunk granularity. Through experiments, it can be seen that by applying the speech recognition method proposed in the embodiments of this specification, different text generation models have strong performance in oral-written performance.

[0140] Experiments on oral-written performance at different model input granularities: Different model input granularities include chapter granularity, chunk granularity and paragraph granularity. Among them, chunk granularity refers to dividing the entire initial recognition text into chunks according to a certain length, and conducting experiments according to chunk granularity. That is to say, in each request, the text generation model must rewrite all paragraphs in the current chunk paragraph by paragraph. Through experiments, it can be seen that there are still certain difficulties in directly performing speech recognition at chapter granularity: the model input length is very long, and when the model output maintains the same order of magnitude as the model input, it brings great difficulty to the model, and there is a significant decline in objective indicators. Therefore, preferably, chunk granularity can be used to cut the initial recognition text into fragments according to length. This can save a lot of tokens compared to paragraph granularity, and can naturally utilize local context information, which is more accurate than chapter granularity. Furthermore, even if paragraph alignment is required, some serial situations are still observed, such as the beginning of the next paragraph being moved to the end of the previous paragraph. This will affect the fidelity of the paragraph, so the model test at the paragraph granularity is not fair. To solve this problem, the embodiment of this specification adds the evaluation of the chunk granularity. In the model test at the paragraph granularity, the performance of the paragraph granularity is slightly better, but in the model test at the chunk granularity, the performance of the chunk granularity is better, thus showing the helpful role of the context.

[0141] Experiments on oral and written performance in different text contexts: For chunk granularity, the paragraphs within a chunk naturally have local context. In order to explicitly compare the role of context, you can choose to conduct experiments at the paragraph-level to compare the helpfulness of local context and global context for speech recognition tasks. Among them, localcontext and global context can both be set to 4 paragraphs. Through experiments, we can see that after the local context is spliced ​​with the global context, the performance is improved, and the performance in chunk-level evaluation is close to that of the chunk granularity, which also proves the auxiliary role of context. After the context is given, the CK-Recall indicator has also been significantly improved, proving that the introduction of context can help the model better understand the full text.

[0142] Experiment on the performance of oral writing of different voice-related texts: Voice-related texts can help eliminate challenging pronunciations and improve the performance of ASR error correction. In the embodiments of this specification, voice-related texts can be used in a variety of ways. In addition to directly providing voice-related text as a model reference for rewriting, a two-stage method can also be used. That is, first summarize the voice-related text, or extract a list of keywords from it, and then give the summary or keyword list as a reference for rewriting. In addition, you can also try the retrieval method to obtain the more relevant content in the summary of the voice-related text as a reference for rewriting. Through experiments, it can be seen that directly giving voice-related text has brought about an improvement in the CK-Recall indicator. Other ways of use have also brought improvements. Among them, the keyword list works better. This shows that the introduction of voice-related text can enhance the recognition of keywords.

[0143] Experiments on oral and written performance in different languages ​​and scenarios: Through experiments, we can see that at the chunk-level granularity, because the model tends to merge some paragraphs for English data rather than keep the paragraphs aligned one by one, the model generally performs better in Chinese than in English, but the model is more capable of maintaining fluency in English. In the podcast scenario, because the podcast introduction provides a lot of voice-related text, the improvement of the podcast scenario CK-Recall index is more obvious, which proves that sufficient voice-related text can help the model's text rewriting.

[0144] Evaluation experiment of large model: Regarding fidelity score: The Spearman rank correlation coefficient between the fidelity evaluation of the large model and the manual evaluation ranking is 0.67, which is close to strong correlation. This shows that the large model is reliable for fidelity judgment and can be used for fidelity report. Regarding written scoring: The experiment found that the correlation between the written scoring and the ranking of manual scoring is not high. Therefore, further research was conducted and it was found that manual scoring tends to give the same ranking, and it is believed that the results of the large model are mostly written. We reviewed the results of different scores given by the large model and found that they were affected by fidelity. The Pearson correlation coefficient between the two is as high as 0.83, showing a very strong correlation. Therefore, the written judgment of the large model is affected by fidelity. Regarding ASR error correction scoring: Similar to the written scoring, it was found that the rank correlation between the ASR error correction score and the manual scoring is not high, and manual scoring tends to give the same ranking. Moreover, the Pearson correlation coefficient between ASR error correction and fidelity scoring is as high as 0.96, showing a very strong correlation. Therefore, the large model is obviously affected by fidelity. Regarding readability scoring: During the experiment, the large model is expected to judge the readability of the results and whether the results are smooth and coherent. The experiment found that readability and written form are very strongly correlated, with a Pearson correlation coefficient of up to 0.97. Therefore, the large model can choose written form for reporting, and there is no need to repeat the readability score. Regarding grammar scoring: The experiment found that the results generated by the large model have almost no grammatical errors, and the grammar score is very high.

[0145] It is worth noting that the speech recognition solution proposed in the embodiments of this specification can use the text generation model to rewrite the initial recognition text in spoken style into the target recognition text in written style, while ensuring that the target recognition text can fully and accurately reflect the original text information of the target speech.

[0146] The following combination Figure 3 , taking the application of the speech recognition method provided in this specification in a conference scenario as an example, the speech recognition method is further described. The speech recognition method provided in the embodiment of this specification can also be applied to other pan-conference scenarios, such as podcast scenarios, interview scenarios, etc. Among them, Figure 3 A flowchart of a conference speech recognition method provided by an embodiment of this specification is shown, which may include the following steps:

[0147] Step 302: Obtain the initial recognition text and speech-associated text of the target conference speech.

[0148] Step 304: Input the initial recognition text and the speech-associated text into a text generation model to obtain a target recognition text of the target conference speech, wherein the text generation model is trained based on sample text pairs and sample-associated texts carried by the sample text pairs, the sample text pairs include initial sample text and target sample text, and the target sample text is a written-style text.

[0149] It should be noted that the implementation of step 302 to step 304 is the same as the implementation of step 102 to step 104, and will not be described in detail in this embodiment of the specification.

[0150] By applying the scheme of the embodiments of this specification, since the target sample text is a written-style text, a text generation model is obtained by training with the target sample text, so that the text generation model has the ability to rewrite the text in a written style. Adding speech-related text to the input of the text generation model deepens the text generation model's understanding of the initial recognition text, ensuring that the target recognition text can comprehensively and accurately reflect the information in the target conference speech, and at the same time realizes the written rewriting of the initial recognition text, thereby enhancing the readability of the target recognition text.

[0151] See also Figure 4 , Figure 4 A flowchart of a speech translation method provided by an embodiment of the present specification is shown, which may include the following steps:

[0152] Step 402: Obtain a target recognition text of a target speech, wherein the target recognition text is generated by a text generation model based on an initial recognition text of the target speech and speech-associated text, and the text generation model is trained based on sample text pairs and sample-associated texts carried by the sample text pairs, and the sample text pairs include an initial sample text and a target sample text, and the target sample text is a text in a written style.

[0153] Step 404: Translate the target recognition text to obtain a text translation result of the target speech.

[0154] It should be noted that the method of generating the target recognition text can refer to the implementation method of step 102 and step 104, and will not be described in detail in the embodiments of this specification.

[0155] In practical applications, the target recognition text is translated, and there are many ways to obtain the text translation result of the target voice, which are specifically selected according to the actual situation. This specification embodiment does not make any limitation to this. In a possible implementation, the application programming interface (API, Application Programming Interface) of the translation tool can be called to translate the target recognition text to obtain the text translation result of the target voice. In another possible implementation, the target recognition text can be input into a text translation model to obtain the text translation result of the target voice. Wherein, the text translation model can be trained based on sample text and the sample translation result of the sample text.

[0156] It is worth noting that the speech recognition method proposed in the embodiments of this specification can not only assist downstream speech translation tasks, but also assist downstream question-answering tasks, summary extraction tasks, etc., and the embodiments of this specification do not impose any limitations on this.

[0157] By applying the scheme of the embodiments of this specification, since the target recognition text is obtained by rewriting the initial recognition text of the target speech in written style based on the speech-associated text by a text generation model with the ability to rewrite the written style of text, the target recognition text can comprehensively and accurately reflect the information in the target speech, and at the same time, it is more fluent in writing and more readable, so that the text translation result obtained by translating the target recognition text is more accurate and more readable.

[0158] See also Figure 5 , Figure 5 A flowchart of a text generation model training method provided by an embodiment of this specification is shown, which may include the following steps:

[0159] Step 502: Obtain a sample text pair, wherein the sample text pair carries a sample associated text, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style.

[0160] Step 504: Input the initial sample text and the sample associated text into the initial generation model to obtain the sample predicted text.

[0161] Step 506: According to the sample prediction text and the target sample text, the model parameters of the initial generation model are adjusted to obtain a trained text generation model.

[0162] It should be noted that the implementation method of step 502 to step 506 is the same as the training method of the above-mentioned text generation model, and will not be repeated in this embodiment of the specification.

[0163] In an optional embodiment of the present specification, the obtaining of the sample text pair may include the following steps:

[0164] Obtaining original recognized text and original associated text of original speech;

[0165] Inputting the original recognition text and the original associated text into the initial generation model to obtain the original predicted text of the original speech;

[0166] The original recognized text is determined as the initial sample text, and the original adjusted text is determined as the target sample text to obtain a sample text pair, wherein the original adjusted text is obtained based on the written style processing of the original predicted text, and the sample associated text carried by the sample text pair is the original associated text.

[0167] Exemplarily, the original speech refers to the speech used to construct the sample text pair. The original recognized text refers to the recognized text of the spoken style expression obtained by ASR of the original speech. The original associated text refers to the text that is associated with the original speech, which can be regarded as the context or auxiliary information of the original speech extension. The multiple sample text pairs obtained in the above manner can constitute a data set for training the initial generation model, which can be called the spoken to written rewriting benchmark data set of ASR text.

[0168] It should be noted that when obtaining the original recognition text and original associated text of the original speech, data from multiple scenarios (such as Chinese and English podcasts, Chinese and English meetings, Chinese and English speeches, etc.) can be collected. For each data, the corresponding auxiliary information is collected as the original associated text. Exemplarily, the original speech in the Chinese and English podcast scene can come from multiple Chinese or English podcast programs involving different topics, and the podcast introduction can be used as the original associated text. The podcast introduction usually includes the name of the guest, professional terms, discussion topics, etc. The original speech in the Chinese and English meeting scene can come from an open source conference data set, and manually annotated topic subtitles, summaries, decisions, to-dos, progress, problems, etc. are selected as the original associated text. The original speech in the Chinese and English speech scene can come from speech programs of different topic categories. Speech meta-information (such as speaker introduction, video title, video category, video description, etc.) can be used as the original associated text.

[0169] After obtaining the original recognition text and original associated text of the original speech, inputting the original recognition text and the original associated text into the initial generation model, and obtaining the original predicted text of the original speech, the original predicted text can be processed in a written style using the data paradigm of spoken style transfer and direct rewriting in GEC to obtain the original adjusted text. The data paradigm of Direct Rewriting usually involves the process of converting sentences containing grammatical errors into the correct form: for the original predicted text, it is expected that the original adjusted text is correct. Correctness here means that the original adjusted text meets the speech recognition standard, such as completing error correction, without obvious ASR and GEC errors, completing the written rewriting of the spoken style text, and converting the spoken style expression into a written style expression completely and appropriately, so that the expression is more fluent and written, the content is faithful to the original text, and there is no obvious spoken style expression.

[0170] In an optional embodiment of the present specification, in terms of data granularity, the original recognized text and the original adjusted text can maintain the consistency of paragraph granularity, so that the original adjusted text can be rewritten between multiple sentences to obtain a more readable result. On the other hand, by considering the paragraph as the minimum relationship that can ensure the one-to-one correspondence between the original recognized text and the original adjusted text, from the perspective of the entire chapter dimension, the text blocks of the original recognized text and the original adjusted text can be divided more flexibly to achieve model training of different granularities.

[0171] By applying the scheme of the embodiments of this specification, the original predicted text is rewritten in a written style to obtain the original adjusted text, the original recognized text is determined as the initial sample text, and the original adjusted text is determined as the target sample text to construct a sample text pair, so that the text generation model trained by using the sample text pair has the ability to rewrite the text in a written style, ensuring that the target recognized text can comprehensively and accurately reflect the information in the target speech, while realizing the written rewriting of the initial recognized text.

[0172] In an optional embodiment of the present specification, after obtaining the sample prediction text, the sample prediction text may be tested to obtain a test result that can reflect the text writing style rewriting ability of the initial generation model, that is, after inputting the initial sample text and the sample associated text into the initial generation model and obtaining the sample prediction text, the following steps may also be included:

[0173] Input the test prompt information, the initial sample text, the sample prediction text and the target sample text into the test model to obtain the test result of the sample prediction text;

[0174] Adjusting the model parameters of the initial generation model according to the sample prediction text and the target sample text in the sample text pair to obtain a trained text generation model may include the following steps:

[0175] When the test result is that the test fails, the model parameters of the initial generation model are adjusted according to the sample prediction text and the target sample text in the sample text pair to obtain a trained text generation model.

[0176] Exemplarily, the test model can be a pre-trained large model. The test prompt information is used to guide the test model to evaluate the written expression of the sample prediction text. The test result refers to the result obtained by the test model to evaluate the written expression of the sample prediction text, and the test result includes test pass and test fail. The test model can have multiple dimensions, such as ASR error correction index, grammar index, fidelity index, written index, readability index, and give a summary of the reasons for generating the test index. In addition, the test model can be tested at paragraph granularity and fragment granularity to ensure the diversity of the test results. In the case of passing the test, it means that the initial generation model has the ability to generate text that meets the speech recognition standard, and there is no need to train the initial generation model. At this time, the initial generation model that has passed the test can be directly used as a text generation model for speech recognition task processing. In the case of a test result of failing the test, it means that the initial generation model does not have the ability to generate text that meets the speech recognition standard. At this time, multiple sample text pairs can be obtained, and the initial generation model can be trained using multiple sample text pairs to obtain a trained text generation model.

[0177] Exemplarily, the test prompt message may be: Assuming that you are a professional text editor, please evaluate the quality of the sample prediction text. The initial sample text comes from ASR and may contain recognition errors, grammatical errors, and colloquial or informal expressions. The initial sample text should eliminate obvious ASR errors and grammatical errors, faithfully preserve the original content, and show a clear written style and good readability. The target sample text can be regarded as one of the qualified rewriting results and can be used as a reference for quality assessment. Please evaluate and score the sample prediction text. The score range for each criterion is 1 to 10, where 1 indicates poor performance and 10 indicates good performance. The returned items include: ASR error correction score: Check whether the sample prediction text effectively corrects the errors in ASR. The higher the score, the more accurate the correction. Grammar score: Evaluate the grammatical accuracy of the sample prediction text. The higher the score, the fewer grammatical errors. Fidelity score: Determine whether the sample prediction text maintains the intent and content of the original speech. The higher the score, the more complete the preservation of the original meaning. Written style score: Evaluate the written language style of the sample prediction text, including the degree of formality and the degree of suitability for written communication. The higher the score, the stronger the written style of the text. Readability score: Evaluate the fluency and overall readability of the sample prediction text, including ease of understanding and naturalness of expression. The higher the score, the better the readability of the text. Evaluation summary: Based on the above scores, briefly explain the reasons for the score, give an overall evaluation of the sample prediction text, and make suggestions for improvement. The following is the initial sample text [initial sample text], the following is the target sample text for reference [target sample text], and the following is the sample prediction text to be evaluated [sample prediction text]. Please provide the score and overall evaluation of each standard in JSON format.

[0178] In practical applications, in addition to using the test model to automatically evaluate the sample prediction text, the initial generation model can also be fully tested by sampling objective indicators or manual evaluation. In terms of objective indicators, the target sample text can be used as a reference to calculate BLUE, ROUGE and the robust indicators for learning text generation (BLEURT). In order to ensure the accuracy of the test, mismatched paragraphs can be skipped. In addition, in order to measure the performance of the model in ASR error correction, the keyword recall (CK-Recall, Challenging Keyword Recall) indicator can be calculated to reflect the recall results of keywords with challenging pronunciation (such as some names and podcast names).

[0179] The scheme of the embodiment of this specification is applied to test the text writing style rewriting ability of the initial generation model, and if the test fails, the initial generation model is trained to ensure that the final text generation model has the ability to rewrite the text writing style.

[0180] By applying the solution of the embodiment of this specification, since the target sample text is a text in a written style, a text generation model is obtained by training with the target sample text, so that the text generation model has the ability to rewrite the text in a written style.

[0181] See also Figure 6 , Figure 6 A flowchart of an information processing method based on a text generation model provided by an embodiment of this specification is shown, which may include the following steps:

[0182] Step 602: Receive a task generation request, wherein the task generation request includes request information.

[0183] Exemplarily, the information processing method based on the text generation model can be applied to a task platform, and the task platform can be deployed on a terminal-side device or a cloud-side device. The task generation request is used to request the generation of task information for the target voice task, and the task generation request represents a request to execute the target voice task. The task generation request usually includes the task type, the expected output format, and the request information. The request information refers to the parameters or descriptive information related to the target voice task carried in the task generation request. The request information is used to request the task platform to generate task information, and the task information is used to execute the target voice task. The request information includes, but is not limited to, the task scenario identifier of the target voice task, the task model identifier, and the sample text pair of the target voice task.

[0184] Step 604: Based on the request information, a text generation model is obtained, wherein the text generation model is trained based on a sample text pair and a sample associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style.

[0185] In actual applications, there are multiple ways to obtain a text generation model based on request information, and the specific method is selected according to the actual situation. The embodiments of this specification do not impose any limitation on this.

[0186] In an optional embodiment of the present specification, the obtaining of the text generation model based on the request information may include the following steps:

[0187] Based on the task scenario identifier, a target scenario template is determined from a plurality of preset scenario templates, and based on the target scenario template, a text generation model is searched from a model library, wherein the model library stores a plurality of text processing models, and the request information includes a task scenario identifier of the target speech task; or,

[0188] Based on the task model identifier, a text generation model is searched from a model library, wherein the request information includes the task model identifier of the target speech task.

[0189] Exemplarily, the task scenario identifier refers to a unique or specific label used to distinguish different task application scenarios. In the embodiment of this specification, the task scenario identifier is a part of the request information. Through the task scenario identifier, a target scenario template matching the request information can be selected from a series of preset scenario templates to generate task information. The preset scenario template is a standard configuration scenario template predefined for different task application scenarios, and each template contains information such as model information matching the task application scenario and task processing flow. Through a series of preset scenario templates, task generation requests for different scenarios can be quickly responded to. Different preset scenario templates correspond to different task types, model information and processing flows. The target scenario template refers to a scenario template that matches the task scenario identifier. When parsing the task generation request, the corresponding target scenario template can be located based on the task scenario identifier, and the corresponding text generation model and other related configuration information can be selected from the model library based on the model information included in the target scenario template. The model library is a resource library that centrally stores text processing models, which are trained and optimized to solve different text processing tasks. Text processing models include but are not limited to text generation models, text translation models, and text summary extraction models. The task model identifier refers to a unique or specific label used to distinguish models applicable to different tasks. The task model identifier of the target speech task refers to the identifier of the model used to process the target speech task.

[0190] By applying the solution of the embodiments of this specification, through pre-defined task scenario templates, task model identifiers and model library resources, the process of acquiring the text generation model is made more flexible, efficient and standardized.

[0191] In another optional embodiment of the present specification, in addition to selecting a text generation model from the model library, the initial processing model in the model library can be trained according to the sample text pair of the target speech task and the sample associated text carried by the sample text pair to obtain the text generation model, that is, the request information includes the sample text pair of the target speech task; the above-mentioned acquisition of the text generation model based on the request information may include the following steps:

[0192] Based on the sample text pairs of the target speech task and the sample associated texts carried by the sample text pairs, the initial generation model is trained to obtain a trained text generation model.

[0193] It should be noted that the implementation method of "training the initial generation model based on the sample text pairs of the target speech task and the sample associated texts carried by the sample text pairs to obtain a trained text generation model" is the same as the training method of the above-mentioned "text generation model", and will not be repeated in this specification embodiment.

[0194] By applying the solution of the embodiments of this specification, the initial generation model is trained based on the sample text pairs of the target speech task and the sample associated texts carried by the sample text pairs, and a trained text generation model is obtained, so that the text generation model is more in line with the task scenario of the target speech task.

[0195] Step 606: Generate task information based on the text generation model, where the task information is used to perform the target speech task.

[0196] Exemplarily, the task information includes the model configuration and processing flow required to perform the target speech task. The terminal device or other server-side components can correctly use the text generation model to process the target speech task based on the task information. The target speech task includes but is not limited to speech recognition tasks and speech error correction tasks.

[0197] It should be noted that when generating task information based on the text generation model, the model parameters of the text generation model can be directly packaged to obtain the task information. Other model information of the text generation model can also be obtained, and task information can be constructed based on the other model information and model parameters, where the other model information includes the processing method of the model input data, the specifications of the expected output results, and the intermediate steps and other auxiliary information that may be involved.

[0198] By applying the solution of the embodiments of this specification, by generating task information of the target voice task, it is possible to reduce system deployment and operation and maintenance costs while ensuring the processing quality and efficiency of the target voice task, and provide users with convenient and efficient voice task processing services.

[0199] See also Figure 7 , Figure 7 A schematic diagram of the structure of a task platform provided by an embodiment of the present specification is shown, and the task platform includes a request interface 702 and a response unit 704;

[0200] The request interface 702 is used to receive a task generation request, wherein the task generation request includes request information;

[0201] The response unit 704 is used to obtain a text generation model based on the request information, wherein the text generation model is trained based on a sample text pair and a sample associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style; based on the text generation model, task information is generated, wherein the task information is used to perform the target speech task.

[0202] In an optional embodiment of the present specification, the task platform further includes a model library, wherein the model library stores a plurality of text processing models;

[0203] A response unit is used to determine a target scene template from multiple preset scene templates based on a task scene identifier, and to search for a text generation model from a model library based on the target scene template, wherein the request information includes the task scene identifier of the target speech task; or, to search for a text generation model from a model library based on a task model identifier, wherein the request information includes the task model identifier of the target speech task.

[0204] By applying the solution of the embodiments of this specification, by generating task information of the target voice task, it is possible to reduce system deployment and operation and maintenance costs while ensuring the processing quality and efficiency of the target voice task, and provide users with convenient and efficient voice task processing services.

[0205] The above is a schematic scheme of a task platform of this embodiment. The technical scheme of the task platform and the technical scheme of the information processing method based on the text generation model are of the same concept. For details not described in detail in the technical scheme of the task platform, please refer to the description of the technical scheme of the information processing method based on the text generation model.

[0206] Corresponding to the above-mentioned speech recognition method embodiment, this specification also provides a speech recognition device embodiment, Figure 8 FIG. 1 shows a schematic diagram of the structure of a speech recognition device provided by an embodiment of the present specification. Figure 8 As shown, the device comprises:

[0207] A first acquisition module 802 is configured to acquire an initial recognition text and a speech-related text of a target speech;

[0208] The first input module 804 is configured to input the initial recognition text and the speech-associated text into the text generation model to obtain the target recognition text of the target speech, wherein the text generation model is trained based on the sample text pairs and the sample-associated texts carried by the sample text pairs, the sample text pairs include the initial sample text and the target sample text, and the target sample text is a written-style text.

[0209] Optionally, the device also includes: a division module, configured to divide the initial recognition text into paragraphs to obtain multiple initial text blocks, wherein the initial text block includes at least one paragraph in the initial recognition text; from the initial recognition text, respectively determine the text context of the multiple initial text blocks; a first input module 804, further configured to input the speech-associated text, the multiple initial text blocks and the text context of the multiple initial text blocks into a text generation model to obtain a target recognition text for the target speech.

[0210] Optionally, the first input module 804 is further configured to repeatedly perform the following steps until target text blocks corresponding to multiple initial text blocks are obtained: a first initial text block is screened out from multiple initial text blocks, wherein the first initial text block is any one of the multiple initial text blocks; the speech-associated text, the first initial text block and the text context of the first initial text block are input into a text generation model to obtain a target text block corresponding to the first initial text block; wherein the target text blocks corresponding to the multiple initial text blocks are used to construct a target recognition text for the target speech.

[0211] Optionally, the division module is further configured to determine the text contexts of the multiple initial text blocks from the initial recognition text respectively according to the positions of the multiple initial text blocks in the initial recognition text and / or the content association relationship between the multiple initial text blocks and the initial recognition text.

[0212] Optionally, the device further includes: a processing module configured to update the speech-associated text according to a text processing strategy, wherein the text processing strategy includes at least one of a summary extraction strategy, a keyword extraction strategy, and a text retrieval strategy.

[0213] Optionally, the device also includes: a sending module configured to send the target recognition text to the client; receive result feedback information sent by the client for the target recognition text; construct model optimization data based on the result feedback information; and use the model optimization data to adjust parameters of the text generation model.

[0214] Optionally, the device also includes: a training module, configured to obtain a sample text pair, wherein the sample text pair carries a sample associated text; input an initial sample text and a sample associated text in the sample text pair into an initial generation model to obtain a sample predicted text; and adjust model parameters of the initial generation model according to the sample predicted text and the target sample text in the sample text pair to obtain a trained text generation model.

[0215] By applying the scheme of the embodiments of this specification, since the target sample text is a written-style text, a text generation model is obtained by training with the target sample text, so that the text generation model has the ability to rewrite the text in a written style. Adding speech-related text to the input of the text generation model deepens the text generation model's understanding of the initial recognition text, ensuring that the target recognition text can comprehensively and accurately reflect the information in the target speech, and at the same time realizes the written rewriting of the initial recognition text, thereby enhancing the readability of the target recognition text.

[0216] The above is a schematic scheme of a speech recognition device of this embodiment. It should be noted that the technical scheme of the speech recognition device and the technical scheme of the above speech recognition method belong to the same concept, and the details not described in detail in the technical scheme of the speech recognition device can be referred to the description of the technical scheme of the above speech recognition method.

[0217] Corresponding to the above-mentioned conference speech recognition method embodiment, this specification also provides a conference speech recognition device embodiment, Fig. 9 FIG. 1 is a schematic diagram showing the structure of a conference speech recognition device provided by an embodiment of the present specification. Fig. 9 As shown, the device comprises:

[0218] The second acquisition module 902 is configured to acquire the initial recognition text and the speech-related text of the target conference speech;

[0219] The second input module 904 is configured to input the initial recognition text and the speech-associated text into the text generation model to obtain the target recognition text of the target conference speech, wherein the text generation model is trained based on the sample text pairs and the sample-associated texts carried by the sample text pairs, the sample text pairs include the initial sample text and the target sample text, and the target sample text is a written-style text.

[0220] By applying the scheme of the embodiments of this specification, since the target sample text is a written-style text, a text generation model is obtained by training with the target sample text, so that the text generation model has the ability to rewrite the text in a written style. Adding speech-related text to the input of the text generation model deepens the text generation model's understanding of the initial recognition text, ensuring that the target recognition text can comprehensively and accurately reflect the information in the target conference speech, and at the same time realizes the written rewriting of the initial recognition text, thereby enhancing the readability of the target recognition text.

[0221] The above is a schematic scheme of a conference speech recognition device of this embodiment. It should be noted that the technical scheme of the conference speech recognition device and the technical scheme of the conference speech recognition method described above are of the same concept, and the details not described in detail in the technical scheme of the conference speech recognition device can be found in the description of the technical scheme of the conference speech recognition method described above.

[0222] Corresponding to the above-mentioned speech translation method embodiment, this specification also provides a speech translation device embodiment, Fig.10 FIG. 1 shows a schematic diagram of the structure of a speech translation device provided by an embodiment of the present specification. Fig.10 As shown, the device comprises:

[0223] The third acquisition module 1002 is configured to acquire a target recognition text of a target speech, wherein the target recognition text is generated by a text generation model based on an initial recognition text of the target speech and speech-related text, and the text generation model is trained based on a sample text pair and a sample-related text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style;

[0224] The translation module 1004 is configured to translate the target recognition text to obtain a text translation result of the target speech.

[0225] By applying the scheme of the embodiments of this specification, since the target recognition text is obtained by rewriting the initial recognition text of the target speech in written style based on the speech-associated text by a text generation model with the ability to rewrite the written style of text, the target recognition text can comprehensively and accurately reflect the information in the target speech, and at the same time, it is more fluent in writing and more readable, so that the text translation result obtained by translating the target recognition text is more accurate and more readable.

[0226] The above is a schematic scheme of a speech translation device of this embodiment. It should be noted that the technical scheme of the speech translation device and the technical scheme of the speech translation method described above are of the same concept, and the details of the technical scheme of the speech translation device not described in detail can be referred to the description of the technical scheme of the speech translation method described above.

[0227] Corresponding to the above text generation model training method embodiment, this specification also provides a text generation model training device embodiment, Fig.11 FIG. 1 is a schematic diagram showing the structure of a text generation model training device provided by an embodiment of the present specification. Fig.11 As shown, the device comprises:

[0228] The fourth acquisition module 1102 is configured to acquire a sample text pair, wherein the sample text pair carries a sample associated text, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style;

[0229] The third input module 1104 is configured to input the initial sample text and the sample associated text into the initial generation model to obtain the sample predicted text;

[0230] The adjustment module 1106 is configured to adjust the model parameters of the initial generation model according to the sample prediction text and the target sample text to obtain a trained text generation model.

[0231] Optionally, the fourth acquisition module 1102 is further configured to obtain the original recognized text and the original associated text of the original speech; input the original recognized text and the original associated text into the initial generation model to obtain the original predicted text of the original speech; determine the original recognized text as the initial sample text, and determine the original adjusted text as the target sample text to obtain a sample text pair, wherein the original adjusted text is obtained based on written style processing of the original predicted text, and the sample associated text carried by the sample text pair is the original associated text.

[0232] Optionally, the device also includes: a testing module, configured to input test prompt information, initial sample text, sample prediction text and target sample text into a test model to obtain a test result of the sample prediction text; a training module, further configured to adjust the model parameters of the initial generation model according to the sample prediction text and the target sample text in the sample text pair when the test result is that the test fails, so as to obtain a trained text generation model.

[0233] By applying the solution of the embodiment of this specification, since the target sample text is a text in a written style, a text generation model is obtained by training with the target sample text, so that the text generation model has the ability to rewrite the text in a written style.

[0234] The above is a schematic scheme of a text generation model training device of this embodiment. It should be noted that the technical scheme of the text generation model training device and the technical scheme of the above-mentioned text generation model training method belong to the same concept, and the details not described in detail in the technical scheme of the text generation model training device can be found in the description of the technical scheme of the above-mentioned text generation model training method.

[0235] Corresponding to the above-mentioned information processing method embodiment based on the text generation model, this specification also provides an information processing device embodiment based on the text generation model, Fig.12 FIG. 1 shows a schematic diagram of the structure of an information processing device based on a text generation model provided by an embodiment of the present specification. Fig.12 As shown, the device comprises:

[0236] The receiving module 1202 is configured to receive a task generation request, wherein the task generation request includes request information;

[0237] The fifth acquisition module 1204 is configured to acquire a text generation model based on the request information, wherein the text generation model is trained based on a sample text pair and a sample associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style;

[0238] The generation module 1206 is configured to generate task information based on the text generation model, wherein the task information is used to perform the target speech task.

[0239] Optionally, the fifth acquisition module 1204 is further configured to determine a target scene template from multiple preset scene templates based on the task scene identifier, and search for a text generation model from a model library based on the target scene template, wherein the model library stores multiple text processing models and the request information includes the task scene identifier of the target speech task; or, search for a text generation model from the model library based on the task model identifier, wherein the request information includes the task model identifier of the target speech task.

[0240] Optionally, the request information includes sample text pairs of the target speech task; the fifth acquisition module 1204 is further configured to train the initial generation model based on the sample text pairs of the target speech task and the sample associated texts carried by the sample text pairs to obtain a trained text generation model.

[0241] By applying the solution of the embodiments of this specification, by generating task information of the target voice task, it is possible to reduce system deployment and operation and maintenance costs while ensuring the processing quality and efficiency of the target voice task, and provide users with convenient and efficient voice task processing services.

[0242] The above is a schematic scheme of an information processing device based on a text generation model of this embodiment. It should be noted that the technical scheme of the information processing device based on a text generation model and the technical scheme of the information processing method based on a text generation model belong to the same concept, and the details not described in detail in the technical scheme of the information processing device based on a text generation model can be found in the description of the technical scheme of the information processing method based on a text generation model.

[0243] Fig.13 13 shows a block diagram of a computing device provided by an embodiment of the present specification. The components of the computing device 1300 include but are not limited to a memory 1310 and a processor 1320. The processor 1320 is connected to the memory 1310 via a bus 1330, and the database 1350 is used to store data.

[0244] The computing device 1300 also includes an access device 1340 that enables the computing device 1300 to communicate via one or more networks 1360. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1340 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0245] In one embodiment of the present specification, the above components of the computing device 1300 and Fig.13 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.13 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0246] The computing device 1300 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 1300 may also be a mobile or stationary server.

[0247] Among them, the processor 1320 is used to execute computer programs / instructions, which, when executed by the processor, implement the steps of the above-mentioned speech recognition method or conference speech recognition method or speech translation method or text generation model training method or information processing method based on text generation model.

[0248] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned speech recognition method, conference speech recognition method, speech translation method, text generation model training method and information processing method based on text generation model belong to the same concept. For details not described in detail in the technical scheme of the computing device, please refer to the description of the technical scheme of the above-mentioned speech recognition method or conference speech recognition method or speech translation method or text generation model training method or information processing method based on text generation model.

[0249] An embodiment of the present specification also provides a computer-readable storage medium storing a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned speech recognition method or conference speech recognition method or speech translation method or text generation model training method or information processing method based on a text generation model.

[0250] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned speech recognition method, conference speech recognition method, speech translation method, text generation model training method and information processing method based on text generation model belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the above-mentioned speech recognition method or conference speech recognition method or speech translation method or text generation model training method or information processing method based on text generation model.

[0251] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned speech recognition method or conference speech recognition method or speech translation method or text generation model training method or information processing method based on a text generation model.

[0252] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the above-mentioned speech recognition method, conference speech recognition method, speech translation method, text generation model training method and information processing method based on text generation model belong to the same concept. For details not described in detail in the technical scheme of the computer program product, please refer to the description of the technical scheme of the above-mentioned speech recognition method or conference speech recognition method or speech translation method or text generation model training method or information processing method based on text generation model.

[0253] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0254] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0255] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0256] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0257] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A speech recognition method, comprising: Obtaining the initial recognition text and speech-related text of the target speech; The initial recognition text and the speech-associated text are input into a text generation model to obtain a target recognition text of the target speech, wherein the text generation model is trained based on a sample text pair and a sample-associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style.

2. The method according to claim 1, before inputting the initial recognition text and the speech-related text into a text generation model to obtain a target recognition text of the target speech, further comprising: Dividing the initial recognized text into paragraphs to obtain a plurality of initial text blocks, wherein the initial text block includes at least one paragraph in the initial recognized text; Determining text contexts of the plurality of initial text blocks respectively from the initial recognized text; The step of inputting the initial recognition text and the speech-associated text into a text generation model to obtain a target recognition text of the target speech comprises: The speech-associated text, the multiple initial text blocks, and the text contexts of the multiple initial text blocks are input into a text generation model to obtain a target recognized text of the target speech.

3. The method according to claim 2, wherein the step of inputting the speech-associated text, the plurality of initial text blocks, and the text contexts of the plurality of initial text blocks into a text generation model to obtain a target recognized text of the target speech comprises: Repeat the following steps until target text blocks corresponding to the multiple initial text blocks are obtained: Filtering out a first initial text block from the multiple initial text blocks, wherein the first initial text block is any one of the multiple initial text blocks; Inputting the speech-associated text, the first initial text block, and the text context of the first initial text block into the text generation model to obtain a target text block corresponding to the first initial text block; The target text blocks corresponding to the multiple initial text blocks are used to construct the target recognition text of the target speech.

4. The method according to claim 2, wherein determining the text contexts of the plurality of initial text blocks from the initial recognized text comprises: According to the positions of the multiple initial text blocks in the initial recognized text and / or the content association relationship between the multiple initial text blocks and the initial recognized text, the text contexts of the multiple initial text blocks are respectively determined from the initial recognized text.

5. The method according to claim 1, before inputting the initial recognition text and the speech-related text into a text generation model to obtain a target recognition text of the target speech, further comprises: The speech-related text is updated according to a text processing strategy, wherein the text processing strategy includes at least one of a summary extraction strategy, a keyword extraction strategy, and a text retrieval strategy.

6. The method according to any one of claims 1 to 5, after inputting the initial recognition text and the speech-related text into a text generation model to obtain a target recognition text of the target speech, further comprising: Sending the target recognition text to the client; Receiving result feedback information sent by the client for the target recognition text; According to the result feedback information, a model is constructed to optimize the data; The model optimization data is used to adjust parameters of the text generation model.

7. The method according to claim 1, before inputting the initial recognition text and the speech-related text into a text generation model to obtain a target recognition text of the target speech, further comprising: Obtaining a sample text pair, wherein the sample text pair carries a sample associated text; Inputting the initial sample text and the sample associated text in the sample text pair into the initial generation model to obtain the sample predicted text; According to the sample prediction text and the target sample text in the sample text pair, the model parameters of the initial generation model are adjusted to obtain a trained text generation model.

8. A conference speech recognition method, comprising: Obtaining the initial recognition text and speech-related text of the target conference speech; The initial recognition text and the speech-associated text are input into a text generation model to obtain a target recognition text of the target conference speech, wherein the text generation model is trained based on a sample text pair and a sample-associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style.

9. A speech translation method, comprising: Acquire a target recognized text of a target speech, wherein the target recognized text is generated by a text generation model based on an initial recognized text of the target speech and speech-associated text, the text generation model is trained based on a sample text pair and a sample-associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style; The target recognition text is translated to obtain a text translation result of the target speech.

10. A text generation model training method, comprising: Acquire a sample text pair, wherein the sample text pair carries a sample associated text, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style; Inputting the initial sample text and the sample associated text into an initial generation model to obtain a sample predicted text; According to the sample prediction text and the target sample text, the model parameters of the initial generation model are adjusted to obtain a trained text generation model.

11. The method according to claim 10, wherein obtaining the sample text pair comprises: Obtaining original recognized text and original associated text of original speech; Inputting the original recognition text and the original associated text into the initial generation model to obtain the original predicted text of the original speech; The original recognized text is determined as the initial sample text, and the original adjusted text is determined as the target sample text to obtain the sample text pair, wherein the original adjusted text is obtained based on the written style processing of the original predicted text, and the sample associated text carried by the sample text pair is the original associated text.

12. The method according to claim 10, after inputting the initial sample text and the sample associated text into the initial generation model to obtain the sample predicted text, further comprising: Input the test prompt information, the initial sample text, the sample prediction text and the target sample text into the test model to obtain the test result of the sample prediction text; The step of adjusting the model parameters of the initial generation model according to the sample prediction text and the target sample text in the sample text pair to obtain a trained text generation model comprises: When the test result is that the test fails, the model parameters of the initial generation model are adjusted according to the sample prediction text and the target sample text in the sample text pair to obtain a trained text generation model.

13. An information processing method based on a text generation model, comprising: receiving a task generation request, wherein the task generation request includes request information; Based on the request information, a text generation model is obtained, wherein the text generation model is trained based on a sample text pair and a sample associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style; Based on the text generation model, task information is generated, wherein the task information is used to perform a target speech task.

14. The method according to claim 13, wherein obtaining a text generation model based on the request information comprises: Based on the task scenario identifier, a target scenario template is determined from a plurality of preset scenario templates, and based on the target scenario template, a text generation model is searched from a model library, wherein the model library stores a plurality of text processing models, and the request information includes the task scenario identifier of the target speech task; or Based on the task model identifier, a text generation model is searched from the model library, wherein the request information includes the task model identifier of the target speech task.

15. The method according to claim 13, wherein the request information includes a sample text pair of a target speech task; The acquiring of a text generation model based on the request information includes: Based on the sample text pairs of the target speech task and the sample associated texts carried by the sample text pairs, the initial generation model is trained to obtain a trained text generation model.

16. A task platform, comprising a request interface and a response unit; The request interface is used to receive a task generation request, wherein: The task generation request includes request information; The response unit is used to obtain a text generation model based on the request information, wherein the text generation model is trained based on a sample text pair and a sample associated text carried by the sample text pair, the sample text pair includes an initial sample text and a target sample text, and the target sample text is a text in a written style; based on the text generation model, task information is generated, wherein the task information is used to perform a target speech task.

17. The task platform according to claim 16 further comprises a model library, wherein: The model library stores a plurality of text processing models; The response unit is used to determine a target scene template from multiple preset scene templates based on a task scene identifier, and search for a text generation model from a model library based on the target scene template, wherein the request information includes the task scene identifier of the target speech task; or, based on a task model identifier, search for a text generation model from the model library, wherein the request information includes the task model identifier of the target speech task.

18. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 7 or claim 8 or claim 9 or any one of claims 10 to 12 or any one of claims 13 to 15 are implemented.

19. A computer-readable storage medium storing a computer program / instruction, which, when executed by a processor, implements the steps of the method described in any one of claims 1 to 7 or claim 8 or claim 9 or any one of claims 10 to 12 or any one of claims 13 to 15.

20. A computer program product comprising a computer program / instructions, which when executed by a processor implements the steps of the method described in any one of claims 1 to 7 or claim 8 or claim 9 or any one of claims 10 to 12 or any one of claims 13 to 15.

Images