An audio script generation method, device, equipment and medium

Abstract

The application provides a recording script generation method, device, equipment and medium, the method comprises the following steps: obtaining a custom pinyin combination sequence; based on the preset mapping relationship between the pinyin sequence and the Chinese character sequence, converting each pinyin in the pinyin combination sequence into the corresponding Chinese character to obtain a Chinese character combination sequence; inputting the Chinese character combination sequence into the trained text generation model to obtain an initial recording script; correcting the initial recording script to obtain a target recording script. According to the actual demand, the pinyin sequence input by the embodiment of the application can be customized to generate a recording script with less text and containing all pinyin types, thereby further reducing the recording corpus required for building a speech synthesis model and reducing the threshold of sound library customization.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of recording technology, and in particular to a recording script generation method, device, equipment and medium. BACKGROUND

[0002] Building a speech synthesis model requires text-to-speech alignment corpus, and the text-to-speech alignment corpus is generally obtained by a dubbing personnel according to a recording script, therefore, the content quality of the recording script directly determines the content quality of the text-to-speech alignment corpus. The recording script is a text file used to guide the dubbing personnel to record, containing the text content to be recorded, however, the present inventors found in the research on the prior art that the method for building the recording script in the prior art either has a large number of recording scripts, or needs professional writers to write the recording script, which is high in cost, and cannot meet the demand of audio library customization through a small amount of corpus. SUMMARY

[0003] The present application provides a recording script generation method, device, equipment and medium, which can reduce the recording corpus required for building a speech synthesis model and reduce the threshold of audio library customization.

[0004] To achieve the above-mentioned purpose, the present application provides a recording script generation method, comprising the following steps:

[0005] obtaining a custom pinyin combination sequence; wherein the pinyin combination sequence comprises a plurality of pinyins and the number of occurrences of each pinyin;

[0006] converting each pinyin in the pinyin combination sequence into a corresponding Chinese character based on a preset mapping relationship between the pinyin sequence and the Chinese character sequence, to obtain a Chinese character combination sequence; wherein the Chinese character combination sequence comprises a plurality of Chinese characters and the number of occurrences of each Chinese character;

[0007] inputting the Chinese character combination sequence into a trained text generation model to obtain an initial recording script;

[0008] correcting the initial recording script to obtain a target recording script.

[0009] As one of the optional embodiments, the step of converting each pinyin in the pinyin combination sequence into a corresponding Chinese character based on a preset mapping relationship between the pinyin sequence and the Chinese character sequence to obtain a Chinese character combination sequence comprises:

[0010] sorting the pinyins in the pinyin combination sequence according to a preset order based on the pinyin arrangement order in the preset mapping table to obtain a sorted pinyin sequence;

[0011] convert the sorted pinyin sequence into a corresponding Chinese character sequence based on the preset mapping table, and add the occurrence number of the pinyin corresponding to each Chinese character in the Chinese character sequence to obtain a Chinese character combination sequence.

[0012] As one of the optional embodiments, the text generation model is trained in the following manner:

[0013] obtain a question and answer pair corpus; wherein the question and answer pair corpus comprises original sentences and question sentences;

[0014] input the question and answer pair corpus into a preset text generation model, train the text generation model, and obtain a trained text generation model.

[0015] As one of the optional embodiments, the question and answer pair corpus is obtained in the following manner:

[0016] obtain a text corpus, and divide the text corpus into multiple original sentences according to a preset sentence division method;

[0017] convert each original sentence in the text corpus into a corresponding pinyin sequence based on a preset mapping table;

[0018] count the occurrence number of each pinyin in each pinyin sequence, and arrange each pinyin in each pinyin sequence according to a preset arrangement order based on the occurrence number to obtain a sorted pinyin sequence;

[0019] convert the sorted pinyin sequence into a corresponding Chinese character sequence based on the preset mapping table, and add the occurrence number of the pinyin corresponding to each Chinese character in the Chinese character sequence to obtain a Chinese character combination sequence.

[0020] construct a question and answer pair by each original sentence and the question sentence corresponding thereto to obtain multiple question and answer pair corpora.

[0021] As one of the optional embodiments, the pinyin combination sequence further comprises the tone of each pinyin;

[0022] Then, the converting each pinyin in the pinyin combination sequence into a corresponding Chinese character based on the preset mapping relationship between the pinyin sequence and the Chinese character sequence to obtain a Chinese character combination sequence comprises:

[0023] arrange each pinyin in the pinyin combination sequence according to a preset order based on the pinyin arrangement order in the preset mapping table to obtain a sorted pinyin sequence;

[0024] Based on the preset mapping table, the pinyin sequence in sequence is converted into a corresponding Chinese character sequence according to the tone of each pinyin, and the number of occurrences of the corresponding pinyin of each Chinese character in the Chinese character sequence is added after the Chinese character to obtain a Chinese character combination sequence.

[0025] As one of the optional embodiments, the preset mapping table includes a general standard Chinese character table.

[0026] As one of the optional embodiments, the text generation model includes GPT, GPT2, GPT3, LaserTagger, and LSTM.

[0027] The embodiment of the present application provides a recording script generation device, which comprises:

[0028] A pinyin combination sequence acquisition module is configured to acquire a customized pinyin combination sequence, wherein the pinyin combination sequence comprises a plurality of pinyins and the number of occurrences of each pinyin.

[0029] A Chinese character combination sequence acquisition module is configured to convert each pinyin in the pinyin combination sequence into a corresponding Chinese character based on a preset mapping relationship between the pinyin sequence and the Chinese character sequence to obtain a Chinese character combination sequence, wherein the Chinese character combination sequence comprises a plurality of Chinese characters and the number of occurrences of each Chinese character.

[0030] An initial recording script acquisition module is configured to input the Chinese character combination sequence into a trained text generation model to obtain an initial recording script.

[0031] A target recording script acquisition module is configured to correct the initial recording script to obtain a target recording script.

[0032] The embodiment of the present application provides a terminal device, which comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to realize the recording script generation method of the above-mentioned embodiment.

[0033] The embodiment of the present application provides a computer readable storage medium, which comprises a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to execute the vehicle positioning method of the above-mentioned embodiment when the computer program runs.

[0034] Compared with the prior art, the recording script generation method, device, equipment and medium provided by the embodiment of the present application can customize the input pinyin sequence according to the actual demand, generate a recording script with less number of characters and containing all pinyin types, thereby further reducing the recording corpus required for building a speech synthesis model and reducing the threshold of sound library customization. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a flowchart of a recording script generation method provided by an embodiment of the present application;

[0036] Figure 2 is a structural diagram of a recording script generation device provided by an embodiment of the present application;

[0037] Figure 3 is a structural diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0039] An embodiment of the present application provides a recording script generation method, referring to Figure 1 is a flowchart of a recording script generation method provided by an embodiment of the present application, and the method comprises steps S11 to S14:

[0040] S11, obtaining a self-defined pinyin combination sequence; wherein the pinyin combination sequence comprises a plurality of pinyins and the number of occurrences of each pinyin.

[0041] It should be noted that the self-defined pinyin combination sequence is in units of sentences.

[0042] S12, converting each pinyin in the pinyin combination sequence into a corresponding Chinese character based on a preset mapping relationship between the pinyin sequence and the Chinese character sequence, to obtain a Chinese character combination sequence; wherein the Chinese character combination sequence comprises a plurality of Chinese characters and the number of occurrences of each Chinese character.

[0043] S13, inputting the Chinese character combination sequence into a trained text generation model to obtain an initial recording script.

[0044] S14, correcting the initial recording script to obtain a target recording script.

[0045] It should be noted that the text correction method includes but is not limited to: a statistical method, a dictionary-based method, and a neural network-based method. In actual application, the method for correcting the recording script can be set according to actual needs or tests, which is not limited herein. It can be understood that if the model-generated text is error-free, the text after text correction is consistent with the original text.

[0046] In addition, it should be noted that text generation is also referred to as natural language generation, and a natural language generation system can be defined as accepting information in a non-language form as input and generating a readable written expression. The present application generates Chinese character text by specifying pinyin text by a user, and then generates Chinese character text. For example, the user specifies the requirement that the pinyin qi3 appears once and lai2 appears once, and by using the embodiment of the present application, the Chinese character text similar to "three months, five buildings, refueling, and laughing" is obtained, and after correction, the Chinese character text meets the preset condition of the user.

[0047] Further, in the prior art, the method for constructing a recording script generally includes: (1) selecting a plurality of written works composed by human beings as a recording script; for example, selecting some articles from primary and secondary school texts, news information, essays, novels, and the like as a recording script; (2) selecting some sentence combinations from sentences composed by human beings as a recording script; for example, dividing news information, essays, novels, and the like into a plurality of sentences, and then setting rules to select some sentences from the divided sentences to construct a recording script; (3) a professional writes a recording script according to preset requirements; for example, specifying a theme, a number of words, a genre, and the like, and then asking a professional writer to write an article according to these preset requirements, and appropriately processing the written article as a recording script. However, the above method has the following problems: (1) selecting a plurality of written works composed by human beings as a recording script; this method constructs a recording script with high literary quality, strict logic, and strong readability, but many pinyins are repeated, and the information redundancy is large, which leads to a large number of recording scripts required to cover all pinyins, and a large number of such corpora required to construct a speech synthesis model, which is not conducive to customizing an audio library with a small amount of resources; (2) selecting some sentence combinations from sentences composed by human beings as a recording script; this method constructs a script with each sentence being logically smooth and strong readability, but the repetition rate of high-frequency pinyins is high, which leads to a large number of scripts required to cover all pinyins, which is not conducive to customizing an audio library; (3) a professional writes a recording script according to preset requirements; this method constructs a script with high literary quality, strong logic, and strong readability, but requires a high level of professional writers, and the cost is high. Moreover, if the number of words and the pinyin coverage requirement are specified, the professional writer needs to have a deep understanding of pinyins, which requires a higher level of professional writers, and it is difficult to successfully implement.

[0048] Compared with the prior art, the recording script generation method provided by the embodiment of the present application can customize the input pinyin sequence according to actual requirements, generate a recording script with a small number of words and containing all pinyin types, thereby further reducing the recording corpus required to construct a speech synthesis model and reducing the threshold of audio library customization. In addition, the present application constructs a recording script in an automatic manner, without consuming human resources, thereby further reducing the writing cost.

[0049] It should be noted that, by using the embodiment of the present application, the number of words of the entire recording script can be reduced in the case that all pinyin types are included and the number of occurrences of each pinyin is higher than a preset threshold, instead of the number of words corresponding to a single sentence pinyin. For example, if a sentence is extracted from a news script to form a recording script, 100,000 words can be needed, and the scheme only needs 10,000 words.

[0050] As one of the optional embodiments, the step S12 comprises:

[0051] S121, based on the pinyin arrangement order in the preset mapping table, the pinyins in the pinyin combination sequence are sorted according to the preset order to obtain a sorted pinyin sequence;

[0052] S122, based on the preset mapping table, the sorted pinyin sequence is converted into a corresponding Chinese character sequence, and the number of occurrences of the pinyin corresponding to each Chinese character is added after the Chinese character in the Chinese character sequence to obtain a Chinese character combination sequence.

[0053] In actual application, the self-defined pinyin combination sequence can be a pinyin combination sequence that has been sorted according to the preset order, and then directly converted based on the preset mapping table to obtain a Chinese character combination sequence, or can be a pinyin combination sequence that has not been sorted in order, and then reordered and converted after input. The specific setting can be set according to actual demand or test, which is not limited here.

[0054] It should be noted that the general composition structure of the self-defined input pinyin combination sequence is: pinyin + number corresponding to tone + number of occurrences, but in actual application, pinyin + number of occurrences can also be used, and the specific setting is related to the effect that the user wants to achieve, which is not limited here.

[0055] For example, several examples of generating a recording script by self-defined pinyin input are as follows:

[0056] (1) Several pinyins occur multiple times:

[0057] Suppose the self-defined pinyins and their numbers of occurrences are: dong 1 wu 1 jia 1 qi 2 yue 2 san 1 you 1 lai 2 xiao 1, the pinyin combination sequence is sorted to obtain qi 2 lai 2 yue 2 jia 1 san 1 you 1 dong 1 wu 1 xiao 1, and then the first one of the Chinese characters corresponding to each pinyin in the "General Standard Chinese Character Table" is found as the exclusive Chinese character of the pinyin, and the Chinese character combination sequence converted is: qi 2 lai 2 yue 2 jia 1 san 1 you 1 dong 1 wu 1 xiao 1;

[0058] (2) Each pinyin occurs only once:

[0059] The pinyin and its occurrence counts are: qi3 1, lai2 1, yue4 1, jia1 1, san1 1, you2 1, dong4 1, wu3 1, xiao4 1. The text input to the conversion model is: qi1 lai1 yue1 jia1 san1 you1 dong1 wu1 xiao1, the text output by the text generation model is: March, five buildings, come on, laugh, and the text of the recorded speech after text correction is: March, five buildings, come on, laugh.

[0060] (3) Multiple occurrences of a single pinyin: <00001i29>

[0061] The pinyin and its occurrence counts are: qi3 4. The text input to the conversion model is: qi4, the text output by the text generation model is: qi qi qi qi, and the text of the recorded speech after text correction is: qi qi qi qi. <0000i32>(4) Multiple occurrences of each pinyin:

[0063] The custom input pinyin and its occurrence counts are: qi3 4, lai2 4, yue4 ४. The text input to the conversion model is: qi4 lai4 yue4, the text output by the text generation model is: yue yue yue up, up up up up up, and the text of the recorded speech after text correction is: yue yue yue up, up up up up up.

[0064] It should be noted that in the actual application process, the preset mapping table can be set according to actual needs or experiments. The reference standard of the general standard Chinese character table here is only for illustrative purposes and is not limited.

[0065] As one of the optional embodiments, the text generation model is trained in the following manner:

[0066] Obtain question-and-answer pair corpora; wherein, the question-and-answer pair corpora include original sentences and question sentences;

[0067] Input the question-and-answer pair corpora into a preset text generation model, train the text generation model, and obtain a trained text generation model.

[0068] [[ID=२8]]As one of the optional embodiments, the question-and-answer pair corpora are obtained in the following manner:

[0069] [[ID=३1]]Obtain text corpora and divide the text corpora into multiple original sentences according to a preset sentence segmentation method; [[ID=३2]] [[ID=३3]]

[0070] [[ID=३4]]Based on a preset mapping table, convert each of the original sentences in the text corpora into a corresponding pinyin sequence; [[ID=३5]]<००००१४९>[[ID=३6]]<००००१५०>counting the number of occurrences of each pinyin in each of the pinyin sequences, and arranging each pinyin in each of the pinyin sequences according to a preset arrangement order according to the number of occurrences, to obtain a sorted pinyin sequence;

[0072] based on a preset mapping table, converting the arranged pinyin sequence into a corresponding Chinese character sequence, and adding the number of occurrences of the corresponding pinyin after each Chinese character in the Chinese character sequence, to obtain a question sentence corresponding to each of the original sentences.

[0073] forming each original sentence and its corresponding question sentence into a set of question and answer pairs, to obtain a plurality of sets of question and answer pair corpora.

[0074] It should be noted that the sentence segmentation method includes but is not limited to: sentence segmentation based on punctuation marks, sentence segmentation using NLP pre-training model.

[0075] For example, the main steps of obtaining or constructing the question and answer pair corpus include:

[0076] 1. Collect a large amount of text corpus of multiple themes; wherein the channels for collecting the text corpus include but are not limited to: news information, network literature, textbooks, essays, ancient literature; the methods for collecting the text corpus include but are not limited to: web crawler, open source download, OCR recognition, speech recognition;

[0077] 2. Convert the text corpus into a question and answer pair corpus;

[0078] (1) Use a sentence segmentation method to divide the corpus into texts in units of sentences, to form a sentence list;

[0079] (2) Each pinyin is represented by a special Chinese character to form a pinyin-Chinese character mapping table;

[0080] 3. Perform Chinese character-to-pinyin operation on each sentence in the sentence list to generate a pinyin sequence; wherein the Chinese character-to-pinyin method includes but is not limited to: Chinese character-to-pinyin based on a pinyin dictionary, Chinese character-to-pinyin based on statistical learning, Chinese character-to-pinyin based on natural language processing pre-training model;

[0081] 4. Generate a question sentence for a question and answer pair;

[0082] (1) Count the number of occurrences of each pinyin in the pinyin sequence converted from each sentence.

[0083] (2) Sort the pinyins in descending order of the number of occurrences of the pinyins, and if the number of occurrences of the pinyins is the same, sort the pinyins in ascending order according to the order of the Universal Standard Chinese Character Table.

[0084] (3) According to the sorted pinyin sequence, convert the pinyin sequence into Chinese characters according to the pinyin-Chinese character mapping table, and add the number of occurrences of the corresponding pinyin after each Chinese character, to form a question sentence for a question and answer pair corpus;

[0085] For example, the original sentences in the sentence list are: Come on, get moving in March and smile in May.

[0086] The generated problem sentence is: Qi 2 Lai 2 Yue 2 Jia 1 San 1 You 1 Dong 1 Wu 1 Xiao 1.

[0087] 5. Generate Q&A pair corpus.

[0088] (1) Use the sentences in the sentence list as answer sentences.

[0089] (2) Combine the problem sentence generated in the previous step and the answer sentence to form a Q&A pair.

[0090] (3) Perform operations on all sentences in the sentence list to generate Q&A pair corpus.

[0091] As an optional embodiment, the pinyin combination sequence further includes the tone of each pinyin; <00...​​​​​​​​​​​​​​

[0097] As one of the optional embodiments, the preset mapping table includes a general specification Chinese character table.

[0098] As one of the optional embodiments, the text generation model includes GPT, GPT2, GPT3, LaserTagger, and LSTM.

[0099] It should be noted that the text generation model can use an open-source pre-trained model as a base model, and the model parameters can also be freely customized, including but not limited to setting parameters according to the amount of training corpus data, setting parameters according to experience, which are not limited herein.

[0100] The embodiment of the present application provides a recording script generation device, which is shown in Figure 2 is a structural schematic diagram of a recording script generation device 20 provided by the embodiment of the present application, which includes:

[0101] The pinyin combination sequence acquisition module 21 is configured to acquire a self-defined pinyin combination sequence; wherein the pinyin combination sequence includes a plurality of pinyins and the number of occurrences of each pinyin.

[0102] The Chinese character combination sequence acquisition module 22 is configured to convert each pinyin in the pinyin combination sequence into a corresponding Chinese character based on a preset mapping relationship between the pinyin sequence and the Chinese character sequence, to obtain a Chinese character combination sequence; wherein the Chinese character combination sequence includes a plurality of Chinese characters and the number of occurrences of each Chinese character.

[0103] The initial recording script acquisition module 23 is configured to input the Chinese character combination sequence into a trained text generation model to obtain an initial recording script.

[0104] The target recording script acquisition module 24 is configured to correct the initial recording script to obtain a target recording script.

[0105] Compared with the prior art, the recording script generation device provided by the embodiment of the present application can customize the input pinyin sequence according to actual needs, generate a recording script with less number of characters and containing all pinyin types, thereby further reducing the recording corpus required for building a speech synthesis model and reducing the threshold of audio library customization.

[0106] As one of the optional embodiments, the Chinese character combination sequence acquisition module 22 is specifically configured to:

[0107] Sort each pinyin in the pinyin combination sequence according to a preset order based on the pinyin arrangement order in the preset mapping table, to obtain a sorted pinyin sequence.

[0108] Based on the preset mapping table, the sorted pinyin sequence is converted into the corresponding Chinese character sequence, and the occurrence count of the corresponding pinyin is added after each Chinese character in the Chinese character sequence to obtain the Chinese character combination sequence.

[0109] As one optional embodiment, the text generation model is trained in the following manner:

[0110] Obtain question-answer pair corpus; wherein, the question-answer pair corpus includes original sentences and question sentences;

[0111] The question-and-answer corpus is input into a preset text generation model, and the text generation model is trained to obtain a trained text generation model.

[0112] As one optional embodiment, the question-answer pair corpus is obtained in the following way:

[0113] Acquire text corpus and divide the text corpus into multiple original sentences according to a preset sentence segmentation method;

[0114] Based on a preset mapping table, each original sentence in the text corpus is converted into a corresponding pinyin sequence;

[0115] The occurrence frequency of each pinyin in each pinyin sequence is counted, and the pinyin in each pinyin sequence is arranged in a preset order according to the occurrence frequency to obtain the sorted pinyin sequence;

[0116] Based on a preset mapping table, the arranged pinyin sequence is converted into a corresponding Chinese character sequence, and the occurrence count of the corresponding pinyin is added after each Chinese character in the Chinese character sequence to obtain the question sentence corresponding to each original sentence.

[0117] Each original sentence and its corresponding question sentence are combined to form a question-answer pair, resulting in multiple sets of question-answer pair corpora.

[0118] As one optional embodiment, the pinyin combination sequence further includes the tone of each pinyin syllable;

[0119] Then, based on the preset mapping relationship between the pinyin sequence and the Chinese character sequence, each pinyin in the pinyin combination sequence is converted into a corresponding Chinese character to obtain a Chinese character combination sequence, including:

[0120] Based on the pinyin arrangement order in the preset mapping table, the pinyin in the pinyin combination sequence is sorted according to the preset order to obtain the sorted pinyin sequence;

[0121] Based on the preset mapping table, the sorted pinyin sequence is converted into a corresponding Chinese character sequence according to the tone of each pinyin, and the number of occurrences of the corresponding pinyin of each Chinese character in the Chinese character sequence is added after the Chinese character to obtain a Chinese character combination sequence.

[0122] As one of the optional embodiments, the preset mapping table includes a general specification Chinese character table.

[0123] As one of the optional embodiments, the text generation model includes GPT, GPT2, GPT3, LaserTagger, and LSTM.

[0124] In addition, it should be noted that the specific implementation schemes and beneficial effects of each embodiment of the recording script generation device provided by the embodiments of the present application correspond to the specific implementation schemes and beneficial effects of each embodiment of the recording script generation method provided by the embodiments of the present application, and will not be repeated here.

[0125] The embodiments of the present application provide a terminal device, referring to Figure 3 is a structural schematic diagram of a terminal device provided by the embodiments of the present application. The terminal device 3 of the embodiment includes a processor 30, a memory 31, and a computer program stored in the memory 31 and executable on the processor 30. The processor 30 implements the recording script generation method of any of the above embodiments when executing the computer program. Alternatively, the processor 30 implements the functions of each module in each device embodiment when executing the computer program.

[0126] For example, the computer program can be divided into one or more modules, which are stored in the memory 31 and executed by the processor 30 to complete the present application. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the terminal device 3.

[0127] The terminal device 3 can be a desktop computer, a notebook computer, a palm computer, and a cloud server, etc. The terminal device 3 can include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art can understand that the schematic diagram is only an example of the terminal device and does not limit the terminal device, which can include more or fewer components than the diagram, or combine certain components, or different components, for example, the terminal device 3 can also include an input / output device, a network access device, a bus, etc.

[0128] The processor 30 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The processor 30 is a control center of the terminal device 3, and is connected with various parts of the terminal device 3 through various interfaces and lines.

[0129] The memory 31 can be used to store computer programs and / or modules. The processor 30 realizes various functions of the terminal device 3 by running or executing the computer programs and / or modules stored in the memory 31, and calling data stored in the memory 31. The memory 31 can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created according to the use of the terminal device 3 (such as audio data, a phone book, etc.), etc. In addition, the memory 31 can include a high-speed random access memory, and can also include a nonvolatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0130] The modules integrated in the terminal device 3, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium, and the computer program can realize the steps of the above-mentioned various method embodiments when executed by the processor 30. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can include any entity or device, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code. It should be noted that the computer readable medium can include or exclude contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0131] It should be noted that the above-described device embodiments are only schematic, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. In addition, the connection relationship between the modules in the device embodiment provided by the present application indicates that there is a communication connection between them, which can be realized as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0132] The embodiment of the present application provides a computer readable storage medium, which comprises a stored computer program, wherein when the computer program runs, the device where the computer readable storage medium is located is controlled to execute the audio script generation method as described above.

[0133] Those skilled in the art will appreciate that modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. Modules or units in the embodiments can be combined or further divided into additional modules or sub-units. Any combination of the disclosed features and / or any method or process or steps of an apparatus of any of the disclosed embodiments can be made, except that at least some of such features and / or processes or steps are mutually exclusive. Each feature disclosed in this specification (including the claims, abstract and drawings) can be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

[0134] It is noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements.

[0135] Furthermore, those skilled in the art will appreciate that the features of the different embodiments can be combined in any combination, which is meant to be within the scope of the application and forms different embodiments. For example, in the claims of the application, any of the embodiments can be used in any combination.

[0136] The preferred embodiments of the application are described above in detail. It should be noted that the above-mentioned embodiments are only used to illustrate the application, but not to limit the application. Those skilled in the art can make some improvements and refinements without departing from the principles of the application. These improvements and refinements are also considered to be within the scope of the application.

Claims

1. A method for generating audio transcripts, characterized in that, include: Obtain a custom pinyin combination sequence; wherein the pinyin combination sequence includes multiple pinyin and the number of times each pinyin appears; Based on a preset mapping relationship between pinyin sequences and Chinese character sequences, each pinyin in the pinyin combination sequence is converted into a corresponding Chinese character to obtain a Chinese character combination sequence; wherein, the Chinese character combination sequence includes multiple Chinese characters and the number of times each Chinese character appears; The Chinese character combination sequence is input into the trained text generation model to obtain the initial audio transcript; The initial audio transcript is corrected to obtain the target audio transcript; The step of converting each pinyin syllable in the pinyin combination sequence into a corresponding Chinese character based on a preset mapping relationship between the pinyin sequence and the Chinese character sequence to obtain a Chinese character combination sequence includes: Based on the pinyin arrangement order in the preset mapping table, the pinyin in the pinyin combination sequence is sorted according to the preset order to obtain the sorted pinyin sequence; Based on the preset mapping table, the sorted pinyin sequence is converted into the corresponding Chinese character sequence, and the occurrence count of the corresponding pinyin is added after each Chinese character in the Chinese character sequence to obtain the Chinese character combination sequence.

2. The method for generating audio transcripts according to claim 1, characterized in that, The text generation model is trained using the following method: Obtain question-answer pair corpus; wherein, the question-answer pair corpus includes original sentences and question sentences; The question-and-answer corpus is input into a preset text generation model, and the text generation model is trained to obtain a trained text generation model.

3. The method for generating audio transcripts according to claim 2, characterized in that, The question-answer pair corpus was obtained through the following methods: Acquire text corpus and divide the text corpus into multiple original sentences according to a preset sentence segmentation method; Based on a preset mapping table, each original sentence in the text corpus is converted into a corresponding pinyin sequence; The occurrence frequency of each pinyin in each pinyin sequence is counted, and the pinyin in each pinyin sequence is arranged in a preset order according to the occurrence frequency to obtain the sorted pinyin sequence; Based on a preset mapping table, the arranged pinyin sequence is converted into a corresponding Chinese character sequence, and the occurrence count of the corresponding pinyin is added after each Chinese character in the Chinese character sequence to obtain the question sentence corresponding to each original sentence; Each original sentence and its corresponding question sentence are combined to form a question-answer pair, resulting in multiple sets of question-answer pair corpora.

4. The method for generating audio transcripts according to claim 1, characterized in that, The pinyin combination sequence also includes the tone of each pinyin syllable; Then, based on the preset mapping relationship between the pinyin sequence and the Chinese character sequence, each pinyin in the pinyin combination sequence is converted into a corresponding Chinese character to obtain a Chinese character combination sequence, including: Based on the pinyin arrangement order in the preset mapping table, the pinyin in the pinyin combination sequence is sorted according to the preset order to obtain the sorted pinyin sequence; Based on the preset mapping table, the sorted pinyin sequence is converted into a corresponding Chinese character sequence according to the tone of each pinyin, and the occurrence count of the corresponding pinyin is added after each Chinese character in the Chinese character sequence to obtain a Chinese character combination sequence.

5. The method for generating audio transcripts according to claim 4, characterized in that, The preset mapping table includes a general standard Chinese character table.

6. The method for generating audio transcripts according to claim 1, characterized in that, The text generation models include GPT, GPT2, GPT3, Laser Tagger, and LSTM.

7. A device for generating audio transcripts, characterized in that, include: A pinyin combination sequence acquisition module is used to acquire a custom pinyin combination sequence; wherein, the pinyin combination sequence includes multiple pinyin and the number of times each pinyin appears; The Chinese character combination sequence acquisition module is used to convert each pinyin in the pinyin combination sequence into a corresponding Chinese character based on a preset mapping relationship between the pinyin sequence and the Chinese character sequence, thereby obtaining a Chinese character combination sequence; wherein, the Chinese character combination sequence includes multiple Chinese characters and the number of times each Chinese character appears; The initial audio transcript acquisition module is used to input the Chinese character combination sequence into the trained text generation model to obtain the initial audio transcript; The target audio transcript acquisition module is used to correct errors in the initial audio transcript to obtain the target audio transcript; The Chinese character combination sequence acquisition module is used for: Based on the pinyin arrangement order in the preset mapping table, the pinyin in the pinyin combination sequence is sorted according to the preset order to obtain the sorted pinyin sequence; Based on the preset mapping table, the sorted pinyin sequence is converted into the corresponding Chinese character sequence, and the occurrence count of the corresponding pinyin is added after each Chinese character in the Chinese character sequence to obtain the Chinese character combination sequence.

8. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the audio transcript generation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the audio transcript generation method as described in any one of claims 1 to 6.

Images