Data generation method and device

Abstract

The embodiment of the invention provides a data generation method and device, and relates to the technical field of voice recognition. The method comprises the steps that a target media file is acquired, the target media file comprises audio content and subtitles, and the audio content comprises target voice with accent; performing voice recognition on the target voice to obtain a first text; acquiring a second text corresponding to the target voice according to the subtitles; obtaining the similarity between the first text and the second text; and if the similarity between the first text and the second text is greater than a preset threshold, generating data according to at least one of the first text and the second text and the target voice.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a method and device for generating data. Background Art

[0002] At present, speech recognition technology has made great achievements in standard speech. However, for speech with an accent, the performance of speech recognition needs to be improved. With the continuous development of speech recognition technology and the expansion of its application scope, improving the recognition performance of speech with an accent becomes more and more important.

[0003] An effective method to improve the recognition performance of accented speech is to add a large amount of accent data to the training data. That is, add a large amount of accented speech and the speech recognition results corresponding to the accented speech to the training data. However, the existing accent data is generally generated by manually recognizing the accented speech and based on the accented speech and its recognition results. The efficiency of generating accent data is very low and difficult to apply on a large scale. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a data generation method and device for improving the efficiency of accent data generation.

[0005] In order to achieve the above purpose, the embodiments of the present application provide the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a method for generating data, including:

[0007] Acquire a target media file, wherein the target media file includes audio content and subtitles, and the audio content includes a target voice with an accent;

[0008] Performing speech recognition on the target speech to obtain a first text;

[0009] Acquire a second text corresponding to the target speech according to the subtitles;

[0010] Obtaining the similarity between the first text and the second text;

[0011] If the similarity between the first text and the second text is greater than a preset threshold, data is generated according to at least one of the first text and the second text and the target voice.

[0012] As an optional implementation of the embodiment of the present application, the step of obtaining the second text corresponding to the target speech according to the subtitles includes:

[0013] Obtaining the timestamp of the target speech;

[0014] Determining a target time period according to the timestamp of the target speech;

[0015] Performing character recognition on the subtitles within the target time period to obtain at least one character string;

[0016] The second text is generated according to the at least one character string.

[0017] As an optional implementation of the embodiment of the present application, the generating the second text according to the at least one character string includes:

[0018] When the at least one character string includes one character string, determining the one character string as the second text;

[0019] When the at least one character string includes a plurality of character strings, the plurality of character strings are merged to obtain the second text.

[0020] As an optional implementation of the embodiment of the present application, the obtaining the similarity between the first text and the second text includes:

[0021] Calculating a word error rate (WER) between the first text and the second text;

[0022] The similarity between the first text and the second text is obtained according to the word error rate between the first text and the second text.

[0023] As an optional implementation of the embodiment of the present application, the generating of data according to at least one of the first text and the second text and the target voice includes:

[0024] Performing accent recognition on the target speech based on an accent model to obtain an accent label of the target speech; wherein the accent model is a model obtained by training a first machine learning model based on a first sample data set, the first sample data set includes a plurality of accent sample data, and any accent sample data includes: a sample speech with an accent and an accent label corresponding to the sample speech with an accent;

[0025] The first text, the target speech, and the accent label of the target speech are determined as a set of data, or the second text, the target speech, and the accent label of the target speech are determined as a set of data.

[0026] As an optional implementation of the embodiment of the present application, before generating data according to at least one of the first text and the second text and the target voice, the method further includes:

[0027] Performing dialect discrimination on the target speech based on a dialect model to determine whether the target speech is a dialect; wherein the dialect model is a model obtained by training a second machine learning model based on a second sample data set, the second sample data set includes a plurality of dialect sample data, and any dialect sample data includes: a sample speech and a dialect label used to characterize whether the sample speech is a dialect;

[0028] If the target speech is a dialect, the first text, the second text and the target speech are discarded.

[0029] As an optional implementation of the embodiment of the present application, before generating data according to at least one of the first text and the second text and the target voice, the method further includes:

[0030] Determining whether the target voice is singing based on audio event detection AED;

[0031] If the target voice is singing, the first text, the second text and the target voice are discarded.

[0032] In a second aspect, an embodiment of the present application provides a data generation device, including:

[0033] an acquisition unit, configured to acquire a target media file, wherein the target media file includes audio content and subtitles, and the audio content includes a target voice with an accent;

[0034] A first recognition unit, configured to perform speech recognition on the target speech to obtain a first text;

[0035] A second recognition unit, configured to obtain a second text corresponding to the target speech according to the subtitles;

[0036] A processing unit, configured to obtain a similarity between the first text and the second text;

[0037] A generating unit is used to generate data according to at least one of the first text and the second text and the target voice when the similarity between the first text and the second text is greater than a preset threshold.

[0038] As an optional implementation of the embodiment of the present application, the second recognition unit is specifically used to obtain the timestamp of the target speech; determine the target time period according to the timestamp of the target speech; perform character recognition on the subtitles within the target time period to obtain at least one character string; and generate the second text according to the at least one character string.

[0039] As an optional implementation of the embodiment of the present application, the second recognition unit is specifically used to, when the at least one string includes one string, determine the one string as the second text; when the at least one string includes multiple strings, merge the multiple strings to obtain the second text.

[0040] As an optional implementation of the embodiment of the present application, the processing unit is specifically used to calculate the word error rate between the first text and the second text; and obtain the similarity between the first text and the second text based on the word error rate between the first text and the second text.

[0041] As an optional implementation of the embodiment of the present application, the generation unit is specifically used to perform accent recognition on the target speech based on an accent model to obtain an accent label of the target speech; wherein the accent model is a model obtained by training a first machine learning model based on a first sample data set, the first sample data set includes multiple accent sample data, and any accent sample data includes: a sample speech with an accent and an accent label corresponding to the sample speech with an accent; and determining the first text, the target speech, and the accent label of the target speech as a group of data, or determining the second text, the target speech, and the accent label of the target speech as a group of data.

[0042] As an optional implementation of the embodiment of the present application, the generation unit is also used to perform dialect discrimination on the target speech based on a dialect model to determine whether the target speech is a dialect before generating data according to at least one of the first text and the second text and the target speech; wherein the dialect model is a model obtained by training a second machine learning model based on a second sample data set, the second sample data set includes multiple dialect sample data, and any dialect sample data includes: a sample speech and a dialect label for characterizing whether the sample speech is a dialect; and in the case that the target speech is a dialect, the first text, the second text and the target speech are discarded.

[0043] As an optional implementation of an embodiment of the present application, the generation unit is also used to determine whether the target speech is singing based on audio event detection before generating data according to at least one of the first text and the second text and the target speech; and in the case that the target speech is singing, discarding the first text, the second text and the target speech.

[0044] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is used to store a computer program and the processor is used to enable the electronic device to implement the data generation method described in any of the above embodiments when executing the computer program.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which, when the computer program is executed by a computing device, enables the computing device to implement any of the above-mentioned data generation methods.

[0046] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to implement any of the above-mentioned methods for generating data.

[0047] The data generation method provided in the embodiment of the present application, after obtaining the target media file including audio content and subtitles, first performs speech recognition on the target speech with an accent in the audio content to obtain a first text, and obtains the second text corresponding to the target speech according to the subtitles, then obtains the similarity between the first text and the second text, and when the similarity between the first text and the second text is greater than a preset threshold, generates data according to at least one of the first text and the second text and the target speech. Since the data generation method provided in the embodiment of the present application can obtain the accent data including the target speech with an accent and the text (first text or second text) corresponding to the target speech with an accent without human intervention, the embodiment of the present application can improve the generation efficiency of the accent data. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 One of the step flow charts of the method for generating data provided in the embodiment of the present application;

[0051] Figure 2 The second step flow chart of the method for generating data provided in the embodiment of the present application;

[0052] Figure 3 A schematic diagram of the structure of a data generation device provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0055] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein. Obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0056] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. In addition, in the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more.

[0057] The existing accent data is generally generated by manually recognizing the accented speech and based on the accented speech and its recognition results. The efficiency of generating accent data is very low and difficult to apply on a large scale. In order to solve the problem of low efficiency in generating accent data, a method and device for generating accent data without manual intervention are provided for the application embodiment. The specific scheme includes the following contents.

[0058] The present application embodiment provides a method for generating data, referring to Figure 1 As shown, the data generation method includes the following steps S11 to S15:

[0059] S11. Obtain target media files.

[0060] The target media file includes audio content and subtitles, and the audio content includes a target voice with an accent.

[0061] The target media file in the embodiment of the present application may be a video file, an audio electronic book, a broadcast file with subtitles, etc., which is not limited in the embodiment of the present application, provided that the target media file includes audio content and subtitles.

[0062] It should be noted that, in addition to audio content and subtitles, the target media file may also include other content. For example, when the target media file is a video file, the target media file may also include image content.

[0063] In the embodiments of the present application, accent refers to the difference in speech pronunciation caused by geographical, social and cultural factors, etc. It is specifically manifested as the change in the degree of modular emphasis on words or special syllables.

[0064] S12: Perform speech recognition on the target speech to obtain a first text.

[0065] In some embodiments, the target speech may be subjected to speech recognition using an automatic speech recognition (ASR) technology to obtain the first text.

[0066] It should be noted that, since the automatic speech recognition technology cannot guarantee that the speech recognition result is completely accurate, and the target speech has an accent, the acquired first text may not be completely identical to the text content of the target speech.

[0067] S13. Acquire a second text corresponding to the target speech according to the subtitles.

[0068] In some embodiments, the subtitles are non-embedded subtitles, and the above step S13 (obtaining the second text corresponding to the target voice according to the subtitles) includes: extracting the subtitle corresponding file data, parsing the subtitle corresponding file data, to obtain the second text corresponding to the target voice.

[0069] In some embodiments, the above step S13 (obtaining the second text corresponding to the target speech according to the subtitles) includes: performing character recognition on the subtitles by optical character recognition (OCR) technology to obtain the second text corresponding to the target speech.

[0070] It should be noted that, affected by factors such as character recognition accuracy, misalignment between subtitles and speech, and errors in subtitles themselves, the acquired second text may not be completely identical to the text content of the target speech.

[0071] S14: Obtain the similarity between the first text and the second text.

[0072] As described above, the first text obtained by performing speech recognition on the target speech may not be completely the same as the text content of the target speech, and the second text obtained according to the subtitles may not be completely the same as the text content of the target speech. If the first text or the second text is directly used as the text content of the target speech to generate accent data, a large amount of erroneous data will be generated in the accent data, which will lead to the difficulty of convergence of the speech recognition model when the speech recognition model is trained based on the generated data. In order to avoid or reduce erroneous data in the accent data on the basis of avoiding manual intervention, the embodiment of the present application obtains the similarity between the first text and the second text, so as to cross-validate the first text and the second text. Only when the same or similar errors occur in the first text and the second text, will erroneous data be caused in the accent data. Therefore, the embodiment of the present application can also reduce or avoid erroneous data in the accent data.

[0073] In some embodiments, the above step S14 (obtaining the similarity between the first text and the second text) includes the following steps a and b:

[0074] Step a: Calculate the word error rate (WER) between the first text and the second text.

[0075] Specifically, word error rate is an important indicator for evaluating speech recognition performance. It is usually used to evaluate the error rate between predicted text and standard text. Therefore, word error rate can also evaluate the similarity between predicted text and standard text. The larger the word error rate, the smaller the similarity between predicted text and standard text; conversely, the smaller the word error rate, the greater the similarity between predicted text and standard text.

[0076] Step b: obtaining the similarity between the first text and the second text according to the word error rate between the first text and the second text.

[0077] In some embodiments, according to the word error rate between the first text and the second text, obtaining the similarity between the first text and the second text includes: calculating the difference between 100% and the word error rate between the first text and the second text to obtain the similarity between the first text and the second text.

[0078] That is, let: the similarity between the first text and the second text be Sim, and the error rate between the first text and the second text be WER, then:

[0079] Sim=100%-WER

[0080] In the above step S14, if the similarity between the first text and the second text is less than or equal to a preset threshold, the first text, the second text and the target speech are discarded, and if the similarity between the first text and the second text is greater than the preset threshold, the following step S15 is executed:

[0081] S15. Generate data according to at least one of the first text and the second text and the target voice.

[0082] In some embodiments, the above step S15 (generating data according to at least one of the first text and the second text and the target voice) includes:

[0083] The first text is determined as the text content of the target speech to generate a set of data.

[0084] In other embodiments, the above-mentioned step S15 (generating data based on at least one of the first text and the second text and the target speech) includes: determining the second text as the text content of the target speech, and generating a set of data based on the target speech and the text content of the target speech.

[0085] In other embodiments, the above step S15 (generating data according to at least one of the first text and the second text and the target speech) includes: obtaining the accuracy of the first text and the second text respectively, and determining the text with higher accuracy in the first text and the second text as the text content of the target speech, and generating a set of data according to the target speech and the text content of the target speech. Among them, the method of obtaining the accuracy of the first text and the second text can be: determining whether the first text or the second text is more in line with the grammatical structure of the natural language based on natural language understanding (NLU), and determining that the text that is more in line with the grammatical structure of the natural language has a higher accuracy.

[0086] In other embodiments, the above-mentioned step S15 (generating data based on at least one of the first text and the second text and the target speech) includes: fusing the first text and the second text to obtain the text content of the target speech, and generating a set of data based on the target speech and the text content of the target speech.

[0087] The data generation method provided in the embodiment of the present application, after obtaining the target media file including audio content and subtitles, first performs speech recognition on the target speech with an accent in the audio content to obtain a first text, and obtains the second text corresponding to the target speech according to the subtitles, then obtains the similarity between the first text and the second text, and when the similarity between the first text and the second text is greater than a preset threshold, generates data according to at least one of the first text and the second text and the target speech. Since the data generation method provided in the embodiment of the present application can obtain the accent data including the target speech with an accent and the text (first text or second text) corresponding to the target speech with an accent without human intervention, the embodiment of the present application can improve the generation efficiency of the accent data.

[0088] As an extension and refinement of the above embodiment, the present application embodiment provides another method for generating data, referring to Figure 2 As shown, the method for generating the data includes the following steps:

[0089] S201: Obtain a target media file.

[0090] The target media file includes audio content and subtitles, and the audio content includes a target voice with an accent.

[0091] S202: Perform speech recognition on the target speech to obtain a first text.

[0092] The implementation of steps S201 and S202 may refer to the implementation of steps S11 and S12 in the above embodiment, and will not be described in detail here to avoid redundancy.

[0093] S203: Obtain the timestamp of the target speech.

[0094] In some embodiments, the timestamp of the target speech may include: a start timestamp of the target speech and an end timestamp of the target speech.

[0095] S204: Determine a target time period according to the timestamp of the target speech.

[0096] In some embodiments, determining the target time period according to the timestamp of the target speech includes: determining the target time period according to the timestamp of the target speech and a preset error. For example, the starting timestamp of the target speech is T s , the end timestamp of the target speech is T e , the preset error is t, then the target time period can be: [T s -t,T e +t].

[0097] S205. Perform character recognition on the subtitles within the target time period to obtain at least one string.

[0098] That is, perform character recognition on the display content of the subtitles within the target time period to obtain one or more strings.

[0099] S206. Generate the second text according to the at least one string.

[0100] In some embodiments, the above step S206 (generating the second text according to the at least one string) includes:

[0101] When the at least one string includes one string, determine the one string as the second text.

[0102] For example: When performing character recognition on the subtitles within the target time period and only obtaining one string, and the content of the string is "Zhuge Liang, styled Kongming, also known as Wolong.", then determine the second text as "Zhuge Liang, styled Kongming, also known as Wolong.".

[0103] In some embodiments, the above step S206 (generating the second text according to the at least one string) further includes:

[0104] When the at least one string includes multiple strings, perform a merging process on the multiple strings to obtain the second text.

[0105] For example: When performing character recognition on the subtitles within the target time period and only obtaining three strings, and the content of the strings are respectively "Zhuge Liang", "styled Kongming", and "also known as Wolong", then the three strings can be merged to determine the second text as "Zhuge Liang, styled Kongming, also known as Wolong.".

[0106] S207. Calculate the word error rate between the first text and the second text.

[0107] In some embodiments, the edit distance (EditDistance) between the first text and the second text can be calculated through algorithms such as the minimum edit distance algorithm and the sparse edit distance algorithm to obtain the word error rate between the first text and the second text.

[0108] Since speech recognition technology often has homophone errors and near-homophone errors. For example: mistakenly recognizing "also known as" as "good", and recognizing "touched" as "dared to move". Combining the context can eliminate similar errors. Therefore, in some embodiments, both the first text and the second text can be converted into pinyin, and then the word error rate between the first text and the second text is calculated.

[0109] S208: Obtain similarity between the first text and the second text according to the word error rate between the first text and the second text.

[0110] As described above, the difference between 100% and the word error rate between the first text and the second text may be calculated to obtain the similarity between the first text and the second text.

[0111] S209: Determine whether the similarity between the first text and the second text is greater than a preset threshold.

[0112] In the embodiment of the present application, the preset threshold can be set according to the error tolerance of the generated accent data. The higher the error tolerance, the smaller the preset threshold can be set, and the lower the error tolerance, the larger the preset threshold needs to be set. For example, the preset threshold can be 90%.

[0113] In the above step S209, if the similarity between the first text and the second text is less than or equal to the preset threshold, the following step S210 is performed:

[0114] S210: discard the first text, the second text, and the target voice.

[0115] Since the similarity between the first text and the second text is less than or equal to the preset threshold, and it is difficult to determine without manual intervention whether the low similarity is caused by inaccurate recognition of the first text or the low similarity is caused by inaccurate recognition of the second text, the corresponding data is directly discarded to reduce or avoid erroneous data in the data.

[0116] In the above step S209, if the similarity between the first text and the second text is greater than the preset threshold, the following steps S211 and S212 are performed:

[0117] S211. Perform accent recognition on the target speech based on an accent model to obtain an accent label of the target speech.

[0118] The accent model is a model obtained by training a first machine learning model based on a first sample data set, wherein the first sample data set includes multiple accent sample data, and any accent sample data includes: a sample speech with an accent and an accent label corresponding to the sample speech with an accent.

[0119] That is, the input of the accent model in the embodiment of the present application is accented speech, and the output of the accent model is the accent label of the accented speech, and the accent label is used to characterize the accent type of the input accented speech.

[0120] In some embodiments, the first machine learning model can be a deep learning neural network model (Deep Neural Network, DNN), a convolutional neural network (Convolutional Neural Networks, CNN) model, a recurrent neural network (Recurrent Neural Network, RNN) model, a fully connected neural network (Full Connect Neural Network, FCN) model, etc., which is not limited to the embodiments of the present application.

[0121] S212: Determine the first text, the target speech, and the accent tag of the target speech as a group of data, or determine the second text, the target speech, and the accent tag of the target speech as a group of data.

[0122] In some embodiments, the data generation method provided in the embodiments of the present application also includes: performing dialect discrimination on the target speech based on a dialect model to determine whether the target speech is a dialect; if the target speech is a dialect, discarding the first text, the second text and the target speech.

[0123] Among them, the dialect model is a model obtained by training a second machine learning model based on a second sample data set, and the second sample data set includes multiple dialect sample data, and any dialect sample data includes: a sample speech and a dialect label used to characterize whether the sample speech is a dialect.

[0124] Dialect and accent are two related but identical concepts. They both involve differences in language variants and phonetic features, but they differ in scope and meaning. Accent refers to differences in the way a specific speech sound is pronounced due to geographical, social and cultural factors. It focuses on the unique way that an individual or group pronounces phonemes, tones and speech rhythms. For example, in English, people in different countries, regions and even cities may have different accents, such as British accent, American accent and so on. Dialect is a broader and more comprehensive language variant than accent. Dialect includes not only phonetic differences, but also differences in vocabulary, grammar and pragmatics. It is a special form of language used in a specific geographical area or a community.

[0125] In some embodiments, the dialect model can be a two-classification model, that is, the input of the dialect model is the speech to be judged, and the output of the dialect model is the dialect label of the speech to be judged, and the dialect label is used to characterize whether the input speech to be judged is a dialect.

[0126] In some embodiments, the dialect model may be a logistic regression model, a support vector machine model, a decision tree model, a random forest model, a gradient boosting tree model, etc., which is not limited in the embodiments of the present application.

[0127] The above embodiment also performs dialect discrimination on the target speech based on the dialect model to determine whether the target speech is a dialect, and discards the first text, the second text and the target speech if the target speech is a dialect, thereby avoiding mixing dialect data into the accent data.

[0128] In some embodiments, the data generation method provided in the embodiments of the present application also includes: determining whether the target speech is singing based on audio event detection (Audio Event Detection, AED); if the target speech is singing, discarding the first text, the second text and the target speech.

[0129] Audio event detection is aimed at detecting whether there are one or more target events (Events) in an audio segment, such as: dog barking, laughter, wind, singing, etc. If data is generated through the above filtering process, the acquired accent data may contain some singing data, thereby affecting the performance of the speech recognition model trained based on the accent data. Therefore, the embodiment of the present application will also filter out the singing data based on audio event detection, thereby avoiding affecting the performance of the speech recognition model trained based on the accent data.

[0130] It should be noted that if the speech recognition model trained with accent data needs to recognize singing content with accent, there is no need to filter out singing data through the above embodiment.

[0131] Based on the same inventive concept, as an implementation of the above method, the embodiment of the present application also provides a server and a client device, which corresponds to the above method embodiment. For ease of reading, this embodiment will no longer repeat the details of the above method embodiment one by one, but it should be clear that the server and client device in this embodiment can correspond to all the contents of the above method embodiment.

[0132] The embodiment of the present application provides a server, Figure 3 is a schematic diagram of the structure of the device for generating the data, such as Figure 3 As shown, the data generating device 300 includes:

[0133] An acquisition unit 31 is used to acquire a target media file, wherein the target media file includes audio content and subtitles, and the audio content includes a target speech with an accent;

[0134] A first recognition unit 32, configured to perform speech recognition on the target speech to obtain a first text;

[0135] A second recognition unit 33, configured to obtain a second text corresponding to the target speech according to the subtitles;

[0136] A processing unit 34, configured to obtain a similarity between the first text and the second text;

[0137] The generating unit 35 is configured to generate data according to at least one of the first text and the second text and the target speech when the similarity between the first text and the second text is greater than a preset threshold.

[0138] As an optional implementation of the embodiment of the present application, the second recognition unit 33 is specifically used to obtain the timestamp of the target speech; determine the target time period according to the timestamp of the target speech; perform character recognition on the subtitles within the target time period to obtain at least one character string; and generate the second text according to the at least one character string.

[0139] As an optional implementation of the embodiment of the present application, the second recognition unit 33 is specifically used to, when the at least one string includes one string, determine the one string as the second text; when the at least one string includes multiple strings, merge the multiple strings to obtain the second text.

[0140] As an optional implementation of the embodiment of the present application, the processing unit 34 is specifically used to calculate the word error rate between the first text and the second text; and obtain the similarity between the first text and the second text based on the word error rate between the first text and the second text.

[0141] As an optional implementation of the embodiment of the present application, the generation unit 35 is specifically used to perform accent recognition on the target speech based on an accent model to obtain an accent label of the target speech; wherein the accent model is a model obtained by training a first machine learning model based on a first sample data set, and the first sample data set includes multiple accent sample data, and any accent sample data includes: a sample speech with an accent and an accent label corresponding to the sample speech with an accent; and determining the first text, the target speech, and the accent label of the target speech as a group of data, or determining the second text, the target speech, and the accent label of the target speech as a group of data.

[0142] As an optional implementation of the embodiment of the present application, the generation unit 35 is also used to perform dialect discrimination on the target speech based on a dialect model to determine whether the target speech is a dialect before generating data according to at least one of the first text and the second text and the target speech; wherein the dialect model is a model obtained by training a second machine learning model based on a second sample data set, the second sample data set includes multiple dialect sample data, and any dialect sample data includes: a sample speech and a dialect label for characterizing whether the sample speech is a dialect; and in the case that the target speech is a dialect, the first text, the second text and the target speech are discarded.

[0143] As an optional implementation of the embodiment of the present application, the generation unit 35 is also used to determine whether the target speech is singing based on audio event detection before generating data according to at least one of the first text and the second text and the target speech; and in the case that the target speech is singing, discarding the first text, the second text and the target speech.

[0144] The data generation device provided in the embodiments of the present application can execute the data generation method provided in any of the above embodiments, and its implementation principle and technical effect are similar, which will not be repeated here.

[0145] Based on the same inventive concept, an embodiment of the present application also provides an electronic device. Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 4 As shown, the electronic device provided in this embodiment includes: a memory 401 and a processor 402, wherein the memory 401 is used to store a computer program, and the processor 402 is used to execute the data generation method provided in the above embodiment when executing the computer program.

[0146] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computing device implements the data generation method provided in the above embodiment.

[0147] Based on the same inventive concept, an embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computing device implements the data generation method provided in the above embodiment.

[0148] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media that include computer-usable program code.

[0149] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0150] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0151] Computer readable media include permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating data, characterized in that: include: Acquire a target media file, wherein the target media file includes audio content and subtitles, and the audio content includes a target voice with an accent; Performing speech recognition on the target speech to obtain a first text; Acquire a second text corresponding to the target speech according to the subtitles; Obtaining the similarity between the first text and the second text; If the similarity between the first text and the second text is greater than a preset threshold, data is generated according to at least one of the first text and the second text and the target voice.

2. The method according to claim 1, characterized in that The step of obtaining a second text corresponding to the target speech according to the subtitles includes: Obtaining the timestamp of the target speech; Determining a target time period according to the timestamp of the target speech; Performing character recognition on the subtitles within the target time period to obtain at least one character string; The second text is generated according to the at least one character string.

3. The method according to claim 2, characterized in that The step of generating the second text according to the at least one character string comprises: When the at least one character string includes one character string, determining the one character string as the second text; When the at least one character string includes a plurality of character strings, the plurality of character strings are merged to obtain the second text.

4. The method according to claim 1, characterized in that: The obtaining the similarity between the first text and the second text includes: Calculating a word error rate between the first text and the second text; The similarity between the first text and the second text is obtained according to the word error rate between the first text and the second text.

5. The method according to claim 1, characterized in that: The generating of data according to at least one of the first text and the second text and the target voice includes: Performing accent recognition on the target speech based on an accent model to obtain an accent label of the target speech; wherein the accent model is a model obtained by training a first machine learning model based on a first sample data set, the first sample data set includes a plurality of accent sample data, and any accent sample data includes: a sample speech with an accent and an accent label corresponding to the sample speech with an accent; The first text, the target speech, and the accent label of the target speech are determined as a set of data, or the second text, the target speech, and the accent label of the target speech are determined as a set of data.

6. The method according to claim 1, characterized in that Before generating data according to at least one of the first text and the second text and the target voice, the method further includes: Performing dialect discrimination on the target speech based on a dialect model to determine whether the target speech is a dialect; wherein the dialect model is a model obtained by training a second machine learning model based on a second sample data set, the second sample data set includes a plurality of dialect sample data, and any dialect sample data includes: a sample speech and a dialect label used to characterize whether the sample speech is a dialect; If the target speech is a dialect, the first text, the second text and the target speech are discarded.

7. The method according to claim 1, characterized in that Before generating data according to at least one of the first text and the second text and the target voice, the method further includes: Determining whether the target speech is singing based on audio event detection; If the target voice is singing, the first text, the second text and the target voice are discarded.

8. A data generating device, characterized in that: include: an acquisition unit, configured to acquire a target media file, wherein the target media file includes audio content and subtitles, and the audio content includes a target voice with an accent; A first recognition unit, configured to perform speech recognition on the target speech to obtain a first text; A second recognition unit, configured to obtain a second text corresponding to the target speech according to the subtitles; A processing unit, configured to obtain a similarity between the first text and the second text; A generating unit is used to generate data according to at least one of the first text and the second text and the target voice when the similarity between the first text and the second text is greater than a preset threshold.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program and the processor is used to enable the electronic device to implement the data generation method described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the method for generating data according to any one of claims 1 to 7.

Images