Data processing method and device

Abstract

The embodiment of the invention provides a data processing method and device, and relates to the technical field of data processing. The method comprises the steps that a target media file is acquired, the target media file comprises audio content and subtitles, and the audio content comprises target voice; performing voice recognition on the target voice to obtain a first text; acquiring a second text corresponding to the target voice according to the subtitles; obtaining an editing distance between the first text and the second text; if the editing distance between the first text and the second text is smaller than or equal to a threshold editing distance, target data are generated, the target data comprise the target voice and a target text, and the target text is generated based on the first text and / or the second text. The embodiment of the invention is used for improving the generation efficiency of the training data of the speech recognition model.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and device. Background Art

[0002] Speech recognition is also known as Automatic Speech Recognition (ASR), and its goal is to convert human speech into text.

[0003] With the development of machine learning technology, speech recognition models based on machine learning models have been widely used in industries such as industry, home appliances, communications, automotive electronics, medical treatment, home services, and consumer electronics. One way to obtain a speech recognition model is to train the machine learning model with a large amount of training data to obtain a speech recognition model. The training data includes speech and text content corresponding to the speech. At present, the training data used to train the speech recognition model is generally generated by manual annotation. That is, by converting the speech into text, and then generating training data based on the speech and the text corresponding to the speech. As mentioned above, a large amount of training data is required in the speech recognition model training process, but the efficiency of generating training data through manual annotation is very low, which is difficult to meet the training needs of the speech recognition model. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a data processing method and device for improving the generation efficiency of data including speech and text content corresponding to the speech.

[0005] In order to achieve the above purpose, the embodiments of the present application provide the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a data processing method, including:

[0007] Acquire a target media file, wherein the target media file includes audio content and subtitles, and the audio content includes a target voice;

[0008] Performing speech recognition on the target speech to obtain a first text;

[0009] Acquire a second text corresponding to the target speech according to the subtitles;

[0010] Obtaining the edit distance between the first text and the second text;

[0011] If the edit distance between the first text and the second text is less than or equal to a threshold edit distance, target data is generated, the target data including: the target speech and target text, the target text being generated based on the first text and / or the second text.

[0012] As an optional implementation of the embodiment of the present application, obtaining the edit distance between the first text and the second text includes:

[0013] The minimum edit distance between the first text and the second text is calculated based on a minimum edit distance algorithm to obtain the edit distance between the first text and the second text.

[0014] As an optional implementation of the embodiment of the present application, obtaining the edit distance between the first text and the second text includes:

[0015] Obtaining a start timestamp and an end timestamp of each character in the second text; the start timestamp and the end timestamp of any character in the second text are respectively a timestamp for starting to display the character and a timestamp for ending to display the character;

[0016] Determining a target time period corresponding to each character in the second text according to a start timestamp and an end timestamp of each character in the second text and a preset offset;

[0017] Determine the editing range corresponding to each character in the second text in the first text according to the target time period corresponding to each character in the second text and the start timestamp and end timestamp of each character in the first text; the start timestamp and end timestamp of any character in the first text are the start timestamp and end timestamp of the speech corresponding to the character, respectively;

[0018] The edit distance between each character in the second text and each character in an edit range corresponding to each character in the second text in the first text is calculated to obtain an edit distance between the first text and the second text.

[0019] As an optional implementation of the embodiment of the present application, obtaining the edit distance between the first text and the second text includes:

[0020] Obtaining the start timestamp and end timestamp of each character in the first text; the start timestamp and end timestamp of any character in the first text are the start timestamp and end timestamp of the speech corresponding to the character respectively;

[0021] Determining a target time period corresponding to each character in the first text according to a start timestamp and an end timestamp of each character in the first text and a preset offset;

[0022] Determine the editing range corresponding to each character in the first text in the second text according to the target time period corresponding to each character in the first text and the start timestamp and end timestamp of each character in the second text; the start timestamp and end timestamp of any character in the second text are the timestamp for starting to display the character and the timestamp for ending to display the character, respectively;

[0023] The edit distance between each character in the first text and each character in an edit range corresponding to each character in the first text in the second text is calculated to obtain an edit distance between the first text and the second text.

[0024] As an optional implementation of the embodiment of the present application, obtaining the edit distance between the first text and the second text includes:

[0025] The first text and the second text are converted into pinyin respectively to obtain a first pinyin text and a second pinyin text;

[0026] The edit distance between the first pinyin text and the second pinyin text is calculated to obtain the edit distance between the first text and the second text.

[0027] As an optional implementation of the embodiment of the present application, before calculating the edit distance between the first phonetic text and the second phonetic text, the method further includes:

[0028] The first pinyin text and the second pinyin text are fuzzified.

[0029] As an optional implementation of the embodiment of the present application, the step of obtaining the second text corresponding to the target speech according to the subtitles includes:

[0030] Obtaining a playback time period of the target voice;

[0031] Character recognition is performed on the display content of the subtitles within the play time period to obtain the second text.

[0032] As an optional implementation of the embodiment of the present application, the method further includes:

[0033] In a case where the edit distance between the first text and the second text is greater than the edit distance, the first text, the second text, and the target voice are discarded.

[0034] In a second aspect, an embodiment of the present application provides a data processing device, including:

[0035] An acquisition unit, configured to acquire a target media file, wherein the target media file includes audio content and subtitles, and the audio content includes a target voice;

[0036] A first recognition unit, configured to perform speech recognition on the target speech to obtain a first text;

[0037] A second recognition unit, configured to obtain a second text corresponding to the target speech according to the subtitles;

[0038] A processing unit, configured to obtain an edit distance between the first text and the second text;

[0039] A generating unit is used to generate target data when the edit distance between the first text and the second text is less than or equal to a threshold edit distance, wherein the target data includes: the target speech and the target text, and the target text is generated based on the first text and / or the second text.

[0040] As an optional implementation of the embodiment of the present application, the processing unit is specifically configured to calculate the minimum edit distance between the first text and the second text based on a minimum edit distance algorithm to obtain the edit distance between the first text and the second text.

[0041] As an optional implementation manner of an embodiment of the present application, the processing unit is specifically used to obtain the start timestamp and end timestamp of each character in the second text; the start timestamp and end timestamp of any character in the second text are respectively the timestamp for starting to display the character and the timestamp for ending to display the character; determine the target time period corresponding to each character in the second text according to the start timestamp and end timestamp of each character in the second text and a preset offset; determine the editing range corresponding to each character in the second text in the first text according to the target time period corresponding to each character in the second text and the start timestamp and end timestamp of each character in the first text; the start timestamp and end timestamp of any character in the first text are respectively the start timestamp and end timestamp of the voice corresponding to the character; calculate the editing distance between each character in the second text and each character in the editing range corresponding to each character in the second text in the first text, so as to obtain the editing distance between the first text and the second text.

[0042] As an optional implementation manner of the embodiment of the present application, the processing unit is specifically used to obtain the start timestamp and end timestamp of each character in the first text; the start timestamp and end timestamp of any character in the first text are respectively the start timestamp and end timestamp of the voice corresponding to the character; determine the target time period corresponding to each character in the first text according to the start timestamp and end timestamp of each character in the first text and a preset offset; determine the editing range corresponding to each character in the first text in the second text according to the target time period corresponding to each character in the first text and the start timestamp and end timestamp of each character in the second text; the start timestamp and end timestamp of any character in the second text are respectively the timestamp for starting to display the character and the timestamp for ending to display the character; calculate the editing distance between each character in the first text and each character in the editing range corresponding to each character in the first text in the second text to obtain the editing distance between the first text and the second text.

[0043] As an optional implementation of an embodiment of the present application, the processing unit is specifically used to convert the first text and the second text into pinyin respectively to obtain a first pinyin text and a second pinyin text; calculate the edit distance between the first pinyin text and the second pinyin text to obtain the edit distance between the first text and the second text.

[0044] As an optional implementation of the embodiment of the present application, the processing unit is also used to perform fuzzy processing on the first pinyin text and the second pinyin text before calculating the edit distance between the first pinyin text and the second pinyin text.

[0045] As an optional implementation of the embodiment of the present application, the second recognition unit is specifically used to obtain the playback time period of the target voice; and perform character recognition on the display content of the subtitles within the playback time period to obtain the second text.

[0046] As an optional implementation of the embodiment of the present application, the generating unit is further configured to discard the first text, the second text and the target speech when the edit distance between the first text and the second text is greater than the edit distance.

[0047] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is used to store a computer program and the processor is used to enable the electronic device to implement the data processing method described in any of the above embodiments when executing the computer program.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which, when the computer program is executed by a computing device, enables the computing device to implement any of the above-mentioned data processing methods.

[0049] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to implement any of the above-mentioned data processing methods.

[0050] The data processing method provided in the embodiment of the present application, after obtaining the target media file including audio content and subtitles, first performs speech recognition on the target speech in the audio content to obtain a first text, and obtains the second text corresponding to the target speech according to the subtitles, and then obtains the edit distance between the first text and the second text, and generates target data including the target speech and the target text when the edit distance between the first text and the second text is less than or equal to the threshold edit distance, and the target text is generated based on the first text and / or the second text. Since the data processing method provided in the embodiment of the present application can obtain the target data including the target speech and the target text (the first text or the second text) without human intervention, the embodiment of the present application can improve the generation efficiency of data including speech and text content corresponding to the speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0053] Figure 1 One of the step flow charts of the data processing method provided in the embodiment of the present application;

[0054] Figure 2 The second step flow chart of the data processing method provided in the embodiment of the present application;

[0055] Figure 3 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein. Obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0059] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. In addition, in the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more.

[0060] At present, the training data of speech recognition models are generally generated by manual standard methods, and the efficiency of generating training data of speech recognition models is very low. In order to solve the problem of low efficiency in generating training data of speech recognition models, a data processing method and device for generating data including speech and text content corresponding to speech without manual intervention is provided for the application embodiment, and the specific scheme includes the following contents.

[0061] The present application embodiment provides a data processing method, referring to Figure 1 As shown, the data processing method includes the following steps S11 to S15:

[0062] S11. Obtain target media files.

[0063] The target media file includes audio content and subtitles, and the audio content includes target speech.

[0064] The target media file in the embodiment of the present application may be a video file, an audio electronic book, a broadcast file with subtitles, etc., which is not limited in the embodiment of the present application, provided that the target media file includes audio content and subtitles.

[0065] It should be noted that the target media file may include multiple voice segments, and the target voice segment may be any one of the multiple voice segments.

[0066] It should also be noted that, in addition to audio content and subtitles, the target media file may also include other content. For example, when the target media file is a video file, the target media file may also include image content.

[0067] S12: Perform speech recognition on the target speech to obtain a first text.

[0068] In some embodiments, the target speech may be subjected to speech recognition using an automatic speech recognition (ASR) technology to obtain the first text.

[0069] It should be noted that, since the automatic speech recognition technology cannot guarantee that the speech recognition result is completely accurate, the acquired first text may not be completely identical to the text content of the target speech.

[0070] S13. Acquire a second text corresponding to the target speech according to the subtitles.

[0071] In some embodiments, the subtitles are non-embedded subtitles, and the above step S13 (obtaining the second text corresponding to the target voice according to the subtitles) includes: extracting the subtitle corresponding file data, parsing the subtitle corresponding file data, to obtain the second text corresponding to the target voice.

[0072] In some embodiments, the above step S13 (obtaining the second text corresponding to the target speech according to the subtitles) includes: performing character recognition on the subtitles by optical character recognition (OCR) technology to obtain the second text corresponding to the target speech.

[0073] It should be noted that, affected by factors such as character recognition accuracy, misalignment between subtitles and speech, and errors in subtitles themselves, the acquired second text may not be completely identical to the text content of the target speech.

[0074] S14. Obtain an edit distance (Edit Distance) between the first text and the second text.

[0075] Edit distance, also known as Levenshtein distance, refers to the minimum number of edit operations required to convert one string into another between two strings, and the permitted edit operations include only three types: 1. Replace one character with another; 2. Insert a character; 3. Delete a character. Generally speaking, the smaller the edit distance, the greater the similarity between the two strings, and vice versa, the larger the edit distance, the smaller the similarity between the two strings.

[0076] As described above, the first text obtained by speech recognition of the target speech may not be exactly the same as the text content of the target speech, and the second text obtained according to the subtitles may not be exactly the same as the text content of the target speech. If the first text or the second text is directly used as the text content of the target speech to generate data, a large number of errors will be caused in the generated data. In order to avoid or reduce errors in data on the basis of avoiding human intervention, the embodiment of the present application obtains the edit distance between the first text and the second text. When there are many errors in the first text or the second text, the edit distance between the first text and the second text will be relatively large, and when there are no errors or fewer errors in the first text and the second text, the edit distance between the first text and the second text will be relatively small. The first text and the second text are cross-validated, so the embodiment of the present application can also reduce or avoid errors in data.

[0077] In the above step S14, if the edit distance between the first text and the second text is greater than the threshold edit distance, the first text, the second text and the target speech are discarded, and if the edit distance between the first text and the second text is less than or equal to the threshold edit distance, the following step S15 is performed:

[0078] S15. Generate target data; the target data includes: the target speech and target text, and the target text is generated based on the first text and / or the second text.

[0079] In some embodiments, the above step S15 (generating target data) includes:

[0080] The first text is determined as the target text, and the target speech and the target text are combined to generate the target data.

[0081] In some other embodiments, the above step S15 (generating target data) includes:

[0082] The second text is determined as the target text, and the target speech and the target text are combined to generate the target data.

[0083] In some other embodiments, the above step S15 (generating target data) includes: obtaining the accuracy of the first text and the second text respectively, and determining the text with higher accuracy in the first text and the second text as the target text, and combining the target speech and the target text to generate the target data. The accuracy of the first text and the second text can be obtained by: determining whether the first text or the second text is more consistent with the grammatical structure of the natural language based on natural language understanding (Natural Language Understanding, NLU), and determining that the text that is more consistent with the grammatical structure of the natural language in the first text and the second text has a higher accuracy.

[0084] In some other embodiments, the above step S15 (generating target data) includes: fusing the first text and the second text to obtain the target text, and combining the target speech and the target text to generate the target data.

[0085] The data processing method provided in the embodiment of the present application, after obtaining the target media file including audio content and subtitles, first performs speech recognition on the target speech in the audio content to obtain a first text, and obtains the second text corresponding to the target speech according to the subtitles, and then obtains the edit distance between the first text and the second text, and generates target data including the target speech and the target text when the edit distance between the first text and the second text is less than or equal to the threshold edit distance, and the target text is generated based on the first text and / or the second text. Since the data processing method provided in the embodiment of the present application can obtain the target data including the target speech and the target text (the first text or the second text) without human intervention, the embodiment of the present application can improve the generation efficiency of data including speech and text content corresponding to the speech.

[0086] As an extension and refinement of the above embodiment, the present application embodiment provides another data processing method, referring to Figure 2 As shown, the data processing method includes the following steps:

[0087] S201: Obtain target media file.

[0088] The target media file includes audio content and subtitles, and the audio content includes target speech.

[0089] S202: Perform speech recognition on the target speech to obtain a first text.

[0090] The implementation of steps S201 and S202 may refer to the implementation of steps S11 and S12 in the above embodiment, and will not be described in detail here to avoid redundancy.

[0091] S203: Obtain a play time period of the target voice.

[0092] Exemplarily, the time when the target voice starts to be played is t1, and the time when the target voice ends to be played is t2, then the playing time period of the target voice is [t1, t2].

[0093] S204: Perform character recognition on the display content of the subtitles in the playback time period to obtain the second text.

[0094] It should be noted that multiple character strings may be displayed within the playback time period. When multiple character strings are identified within the playback time period, the multiple character strings may be merged to obtain the second text.

[0095] S205: Obtain an edit distance between the first text and the second text.

[0096] In some embodiments, the above step S205 (obtaining the edit distance between the first text and the second text) includes:

[0097] The minimum edit distance (Minimum Edit Distance) between the first text and the second text is calculated based on a minimum edit distance algorithm to obtain the edit distance between the first text and the second text.

[0098] In some embodiments, calculating the minimum edit distance between the first text and the second text based on the minimum edit distance algorithm comprises the following steps:

[0099] Step 1: Create a table of (m+2)*(n+2).

[0100] Here, m and n are the number of characters in the first text and the number of characters in the second text respectively.

[0101] Exemplarily, when the first text is "Zhuge Liang, courtesy name Kongming, and nickname Wolong", and the second text is "Zhuge Liang, courtesy name Kongming, and nickname Wolong", the table created in step 1 above may be as shown in Table 1 below:

[0102] Table 1

[0103] various Kudzu Bright Character hole bright good Lying dragon various Kudzu Bright Character hole bright Number Lying dragon

[0104] Step 2: Fill in the serial number of each character in the first text below each character in the first text, and fill in the serial number of each character in the second text to the right of each character in the second text.

[0105] As described in the above example, after performing the above step 2 on Table 1, the resulting table can be as shown in Table 2 below:

[0106] Table 2

[0107] various Kudzu Bright Character hole bright good Lying dragon 0 1 2 3 4 5 6 7 8 9 various 1 Kudzu 2 Bright 3 Character 4 hole 5 bright 6 Number 7 Lying 8 dragon 9

[0108] Step 3: For the remaining blanks in Table 2, obtain the first value, the second value, and the third value, and fill the minimum value among the first value, the second value, and the third value into the corresponding blank.

[0109] Among them, if the character on the far left of the blank (the character in the second text) is the same as the character directly above the blank (the character in the first text), then the first value is the value in the table entry diagonally above and to the left adjacent to the blank; if the character on the far left of the blank is different from the character directly above the blank, then the first value is the value in the table entry diagonally above and to the left adjacent to the blank plus one. The second value is the value in the table entry adjacent to the left of the blank plus one, and the third value is the value in the table entry adjacent to the above of the blank plus one.

[0110] For example: For the blank in the 3rd row and 3rd column of Table 2, since the character on the far left of this blank and the character directly above it are both "Zhu", the first value of this blank is 0, the second value is 2, and the third value is 2. The blank in the 3rd row and 3rd column of Table 2 should be filled with 0.

[0111] Another example: For the blank in the 3rd row and 4th column of Table 2, since the character on the far left is "Zhu" while the character directly above is "Ge", the first value of this blank is 2, the second value is 1, and the third value is 3. The blank in the 3rd row and 4th column of Table 2 should be filled with 1.

[0112] As described in the above example, after performing the above step 3 on Table 2, the resulting table can be as shown in Table 3 below:

[0113] Table 3

[0114] various Kudzu Bright Character hole bright good Lying dragon 0 1 2 3 4 5 6 7 8 9 various 1 0 1 2 3 4 5 6 7 8 Kudzu 2 1 0 1 2 3 4 5 6 7 Bright 3 2 1 0 1 2 3 4 5 6 Character 4 3 2 1 0 1 2 3 4 5 hole 5 4 3 2 1 0 1 2 3 4 bright 6 5 4 3 2 1 0 1 2 3 Number 7 6 5 4 3 2 1 1 2 3 Lying 8 7 6 5 4 3 2 2 1 2 dragon 9 8 7 6 5 4 3 3 2 1

[0115] Step 4: Determine the number in the bottom - right - most table entry of the table as the minimum edit distance between the first text and the second text.

[0116] As described in the above example, the minimum edit distance between the first text and the second text is 1. That is, between the first text and the second text, the minimum number of edit operations required to convert one string to the other is 1 time. Specifically: Replace the "hao" in the first text with the "hao" in the second text, and the first text can be converted to the second text.

[0117] Although the minimum edit distance algorithm can calculate the minimum edit distance between two texts, the time complexity and space complexity of the minimum edit distance algorithm increase quadratically with the increase of the text length. When the text length is long, it is likely that the minimum edit distance algorithm cannot be used to calculate the edit distance between the texts due to excessive calculation. Considering the temporal correlation between the characters in the first text and the characters in the second text, the edit distance between the first text and the second text can also be obtained through the following two schemes.

[0118] Solution 1:

[0119] Step 1) Obtain the start timestamp and end timestamp of each character in the second text.

[0120] The start timestamp and the end timestamp of any character in the second text are respectively the timestamp of starting to display the character and the timestamp of ending to display the character.

[0121] Exemplarily, when the second text is "Zhuge Liang, courtesy name Kongming, alias Wulong", the timestamp when the character string "Zhuge Liang" starts to be displayed is 0.5, the timestamp when the character string "Zhuge Liang" ends to be displayed is 1.5, the timestamp when the character string "custom name Kongming" starts to be displayed is 2.0, the timestamp when the character string "custom name Kongming" ends to be displayed is 3.0, the timestamp when the character string "alias Wulong" starts to be displayed is 3.5, and the timestamp when the character string "alias Wulong" ends to be displayed is 4.0, then the start timestamp and end timestamp of the characters "Zhu", "Ge", and "Liang" in the second text can be obtained as 0.5 and 1.5; the start timestamp and end timestamp of the characters "custom name", "Kong", and "Ming" in the second text are 2.0 and 3.0; the start timestamp and end timestamp of the characters "alias", "Wo", and "Long" in the second text are 3.4 and 4.0.

[0122] Step 2) determining a target time period corresponding to each character in the second text according to a start timestamp and an end timestamp of each character in the second text and a preset offset.

[0123] The start timestamp and the end timestamp of any character in the first text are respectively the start timestamp and the end timestamp of the speech corresponding to the character.

[0124] In some embodiments, the target time period corresponding to each character in the second text is determined based on the start timestamp and end timestamp of each character in the second text and a preset offset, including: determining the difference between the start timestamp of the character and the preset offset as the starting point of the corresponding target time period of the character, and determining the sum of the end timestamp of the character and the preset offset as the end point of the corresponding target time period of the character.

[0125] For example, when the preset offset is 0.6, since the start timestamp and end timestamp of the characters "Zhu", "Ge", and "Liang" in the second text are 0.5 and 1.5 respectively, it can be determined that the target time periods corresponding to the characters "Zhu", "Ge", and "Liang" in the second text are all [0.0, 2.1].

[0126] Step 3): Determine the editing range corresponding to each character in the second text in the first text according to the target time period corresponding to each character in the second text and the start timestamp and end timestamp of each character in the first text.

[0127] Among them, the start timestamp and end timestamp of any character in the first text are the start timestamp and end timestamp of the voice corresponding to that character respectively.

[0128] Continuing with the above example, when the first text is "Zhuge Liang, styled Kongming, is fond of Wolong", the start timestamps and end timestamps of each character in the first text are as follows:

[0129] ["Zhu", 0.4, 0.7];

[0130] ["Ge", 0.8, 1.2];

[0131] ["Liang", 1.3, 1.6];

[0132] ["styled", 1.8, 2.2];

[0133] ["Kong", 2.3, 2.8];

[0134] ["ming", 2.9, 3.2];

[0135] ["is fond of", 3.3, 3.5];

[0136] ["Wo", 3.6, 4.0];

[0137] ["long", 4.1, 4.2];

[0138] Since the target time period corresponding to the character "Zhu" in the second text is [0.0, 2.1], and from the start timestamps and end timestamps of the characters "Zhu", "Ge", and "Liang" in the first text, it can be seen that the characters "Zhu", "Ge", "Liang", and "styled" in the first text are within the target time period corresponding to the character "Zhu" in the second text. Therefore, it is determined that the editing range corresponding to the character "Zhu" in the second text in the first text is [Zhu, Ge, Liang, styled].

[0139] Step 4), calculate the edit distance between each character in the second text and each character in the edit range corresponding to each character in the second text in the first text, so as to obtain the edit distance between the first text and the second text.

[0140] As described in the above example, when the preset offset is 0.6, it can be determined that the edit ranges corresponding to the characters "Zhu", "Ge", and "Liang" in the second text in the first text are [Zhu, Ge, Liang, Zi], and the edit ranges corresponding to the characters "Zi", "Kong", and "Ming" in the second text in the first text are [Zi, Kong, Ming, Hao], and the edit ranges corresponding to the characters "Hao", "Wo", and "Long" in the second text in the first text are [Hao, Wo, Long]; when calculating the edit distance between each character in the second text and each character in the edit range corresponding to each character in the second text in the first text, only the values of the table entries with backgrounds in Table 4 below need to be calculated:

[0141] Table 4

[0142]

[0143] Compared with the minimum edit distance algorithm that needs to calculate the values of 9 * 9 = 81 table entries, the above embodiment only needs to calculate the values of 4 * 3 + 4 * 3 + 3 * 3 = 33 table entries. The above embodiment can greatly reduce the space complexity and time complexity when calculating the edit distance between the first text and the second text, and further improve the generation efficiency of the training data of the speech recognition model.

[0144] Solution Two

[0145] Step ①, obtain the start timestamp and end timestamp of each character in the first text.

[0146] Wherein, the start timestamp and end timestamp of any character in the first text are respectively the start timestamp and end timestamp of the voice corresponding to the character.

[0147] Step ②, determine the target time period corresponding to each character in the first text according to the start timestamp and end timestamp of each character in the first text and the preset offset.

[0148] In some embodiments, determining the target time period corresponding to each character in the first text according to the start timestamp and end timestamp of each character in the first text and the preset offset includes: determining the difference between the start timestamp of the character and the preset offset as the start point of the target time period corresponding to the character, and determining the sum of the end timestamp of the character and the preset offset as the end point of the target time period corresponding to the character.

[0149] Step ③: Determine the editing range corresponding to each character in the first text in the second text according to the target time period corresponding to each character in the first text and the start timestamp and end timestamp of each character in the second text.

[0150] Step 4): Calculate the edit distance between each character in the first text and each character in the editing range corresponding to each character in the first text in the second text to obtain the edit distance between the first text and the second text.

[0151] For example: If the first text is "Zhuge Liang, styled Kongming, fond of Wolong", then the start timestamp and end timestamp of each character in the first text are: ["Zhu", 0.4, 0.7]; ["Ge", 0.8, 1.2]; ["Liang", 1.3, 1.6]; ["styled", 1.8, 2.2]; ["Kong", 2.3, 2.8]; ["Ming", 2.9, 3.2]; ["fond", 3.3, 3.5]; ["of", 3.6, 4.0]; ["Wolong", 4.1, 4.2]. The second text is "Zhuge Liang, styled Kongming, named Wolong". The start timestamp and end timestamp of the characters "Zhu", "Ge", and "Liang" in the second text are 0.5 and 1.5; the start timestamp and end timestamp of the characters "styled", "Kong", and "Ming" in the second text are 2.0 and 3.0; the start timestamp and end timestamp of the characters "named", "of", and "Wolong" in the second text are 3.4 and 4.0. When the preset offset is 0.6, it can be determined that the target time period corresponding to "Zhu" in the first text is [0.0, 1.3], and the corresponding editing range in the second text is [Zhu, Ge, Liang]; the target time period corresponding to "Ge" in the first text is [0.2, 1.8], and the corresponding editing range in the second text is [Zhu, Ge, Liang]; the target time period corresponding to "Liang" in the first text is [0.7, 2.2], and the corresponding editing range in the second text is [Zhu, Ge, Liang, styled, of, Wolong]; the target time period corresponding to "styled" in the first text is [1.2, 2.8], and the corresponding editing range in the second text is [Zhu, Ge, Liang, styled, of, Wolong]; the target time period corresponding to "Kong" in the first text is [1.7, 3.5], and the corresponding editing range in the second text is [styled, of, Wolong, named, Kong, Ming]. The calculation method of the editing range corresponding to other characters in the first text in the second text is similar and will not be elaborated here. To calculate the edit distance between each character in the first text and each character in the editing range corresponding to each character in the first text in the second text, only the values of the table entries with backgrounds in Table 5 below need to be calculated:

[0152] Table 5

[0153]

[0154] Compared with the need to calculate the values of 9 * 9 = 81 table entries in the minimum edit distance algorithm, the above embodiment only needs to calculate the values of 4 * 3 + 5 * 3 + 4 * 3 = 39 table entries. The above embodiment can also greatly reduce the space complexity and time complexity when calculating the edit distance between the first text and the second text, thereby further improving the generation efficiency of the training data of the speech recognition model.

[0155] S206. Determine whether the edit distance between the first text and the second text is less than or equal to the threshold edit distance.

[0156] In the above step S206, if the edit distance between the first text and the second text is greater than the threshold edit distance, then perform the following step S207:

[0157] S207. Discard the first text, the second text, and the target speech.

[0158] Since the edit distance between the first text and the second text is greater than the threshold edit distance, there are a large number of errors in the first text and / or the second text, and it is difficult to determine whether the error is caused by inaccurate recognition of the first text or inaccurate recognition of the second text without manual intervention. Therefore, the corresponding data is directly discarded to reduce or avoid errors in the finally generated data.

[0159] In the above step S206, if the edit distance between the first text and the second text is less than or equal to the threshold edit distance, then perform the following step S208:

[0160] S208. Generate target data; the target data includes: the target speech and the target text, and the target text is generated based on the first text and / or the second text.

[0161] Since speech recognition technology often has homophone errors and near-homophone errors, for example: mistakenly recognizing "号" as "好", and "感动" as "敢动", and combining the context of the text can eliminate similar errors. Therefore, in some embodiments, obtaining the edit distance between the first text and the second text further includes: converting the first text and the second text into pinyin respectively to obtain the first pinyin text and the second pinyin text; calculating the edit distance between the first pinyin text and the second pinyin text to obtain the edit distance between the first text and the second text.

[0162] In some embodiments, the data processing method provided by the embodiments of the present application also includes: before calculating the edit distance between the first pinyin text and the second pinyin text, fuzzifying the first pinyin text and the second pinyin text.

[0163] In some embodiments, the fuzzification process is performed on the first pinyin text and the second pinyin text, including: uniformly rewriting the retroflex sounds in the first pinyin text and the second pinyin text into flat tongue sounds. For example: modifying "zh" to "z", modifying "ch" to "c", and modifying "sh" to "s".

[0164] In some embodiments, the first phonetic text and the second phonetic text are fuzzified, including: changing the easily confused initial consonants in the first phonetic text and the second phonetic text to the same initial consonant. For example, all initial consonants "n" are changed to "l".

[0165] Based on the same inventive concept, as an implementation of the above method, the embodiment of the present application also provides a server and a client device, which corresponds to the above method embodiment. For ease of reading, this embodiment will no longer repeat the details of the above method embodiment one by one, but it should be clear that the server and client device in this embodiment can correspond to all the contents of the above method embodiment.

[0166] The embodiment of the present application provides a server, Figure 3 is a structural diagram of the data processing device, such as Figure 3 As shown, the data processing device 300 includes:

[0167] An acquisition unit 31 is used to acquire a target media file, wherein the target media file includes audio content and subtitles, and the audio content includes a target voice;

[0168] A first recognition unit 32, configured to perform speech recognition on the target speech to obtain a first text;

[0169] A second recognition unit 33, configured to obtain a second text corresponding to the target speech according to the subtitles;

[0170] A processing unit 34, configured to obtain an edit distance between the first text and the second text;

[0171] The generating unit 35 is used to generate target data when the edit distance between the first text and the second text is less than or equal to a threshold edit distance, wherein the target data includes: the target speech and the target text, and the target text is generated based on the first text and / or the second text.

[0172] As an optional implementation of the embodiment of the present application, the processing unit 34 is specifically configured to calculate the minimum edit distance between the first text and the second text based on a minimum edit distance algorithm to obtain the edit distance between the first text and the second text.

[0173] As an optional implementation manner of the embodiment of the present application, the processing unit 34 is specifically used to obtain the start timestamp and end timestamp of each character in the second text; the start timestamp and end timestamp of any character in the second text are respectively the timestamp for starting to display the character and the timestamp for ending to display the character; determine the target time period corresponding to each character in the second text according to the start timestamp and end timestamp of each character in the second text and a preset offset; determine the editing range corresponding to each character in the second text in the first text according to the target time period corresponding to each character in the second text and the start timestamp and end timestamp of each character in the first text; the start timestamp and end timestamp of any character in the first text are respectively the start timestamp and end timestamp of the voice corresponding to the character; calculate the editing distance between each character in the second text and each character in the editing range corresponding to each character in the second text in the first text, so as to obtain the editing distance between the first text and the second text.

[0174] As an optional implementation manner of the embodiment of the present application, the processing unit 34 is specifically used to obtain the start timestamp and end timestamp of each character in the first text; obtain the start timestamp and end timestamp of each character in the first text; the start timestamp and end timestamp of any character in the first text are respectively the start timestamp and end timestamp of the voice corresponding to the character; determine the target time period corresponding to each character in the first text according to the start timestamp and end timestamp of each character in the first text and the preset offset; determine the editing range corresponding to each character in the first text in the second text according to the target time period corresponding to each character in the first text and the start timestamp and end timestamp of each character in the second text; the start timestamp and end timestamp of any character in the second text are respectively the timestamp for starting to display the character and the timestamp for ending to display the character; calculate the editing distance between each character in the first text and each character in the editing range corresponding to each character in the first text in the second text to obtain the editing distance between the first text and the second text.

[0175] As an optional implementation of an embodiment of the present application, the processing unit 34 is specifically used to convert the first text and the second text into pinyin respectively to obtain a first pinyin text and a second pinyin text; calculate the edit distance between the first pinyin text and the second pinyin text to obtain the edit distance between the first text and the second text.

[0176] As an optional implementation of the embodiment of the present application, the processing unit 34 is also used to perform fuzzy processing on the first pinyin text and the second pinyin text before calculating the edit distance between the first pinyin text and the second pinyin text.

[0177] As an optional implementation of the embodiment of the present application, the second recognition unit 33 is specifically used to obtain the playback time period of the target voice; and perform character recognition on the display content of the subtitles within the playback time period to obtain the second text.

[0178] As an optional implementation of the embodiment of the present application, the generating unit 35 is further configured to discard the first text, the second text and the target speech when the edit distance between the first text and the second text is greater than the threshold edit distance.

[0179] The data processing device provided in the embodiment of the present application can execute the data processing method provided in any of the above embodiments. Its implementation principle and technical effect are similar and will not be repeated here.

[0180] Based on the same inventive concept, an embodiment of the present application also provides an electronic device. Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 4 As shown, the electronic device provided in this embodiment includes: a memory 401 and a processor 402, wherein the memory 401 is used to store a computer program, and the processor 402 is used to execute the data processing method provided in the above embodiment when executing the computer program.

[0181] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computing device implements the data processing method provided in the above embodiment.

[0182] Based on the same inventive concept, an embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computing device implements the data processing method provided in the above embodiment.

[0183] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media that include computer-usable program code.

[0184] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0185] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0186] Computer readable media include permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, characterized in that: include: Acquire a target media file, wherein the target media file includes audio content and subtitles, and the audio content includes a target voice; Performing speech recognition on the target speech to obtain a first text; Acquire a second text corresponding to the target speech according to the subtitles; Obtaining the edit distance between the first text and the second text; If the edit distance between the first text and the second text is less than or equal to a threshold edit distance, target data is generated, the target data including: the target speech and target text, the target text being generated based on the first text and / or the second text.

2. The method according to claim 1, characterized in that The obtaining the edit distance between the first text and the second text includes: The minimum edit distance between the first text and the second text is calculated based on a minimum edit distance algorithm to obtain the edit distance between the first text and the second text.

3. The method according to claim 1, characterized in that The obtaining the edit distance between the first text and the second text includes: Obtaining a start timestamp and an end timestamp of each character in the second text; the start timestamp and the end timestamp of any character in the second text are respectively a timestamp for starting to display the character and a timestamp for ending to display the character; Determining a target time period corresponding to each character in the second text according to a start timestamp and an end timestamp of each character in the second text and a preset offset; Determine the editing range corresponding to each character in the second text in the first text according to the target time period corresponding to each character in the second text and the start timestamp and end timestamp of each character in the first text; the start timestamp and end timestamp of any character in the first text are the start timestamp and end timestamp of the speech corresponding to the character, respectively; The edit distance between each character in the second text and each character in an edit range corresponding to each character in the second text in the first text is calculated to obtain an edit distance between the first text and the second text.

4. The method according to claim 1, characterized in that The obtaining the edit distance between the first text and the second text includes: Obtaining the start timestamp and end timestamp of each character in the first text; the start timestamp and end timestamp of any character in the first text are the start timestamp and end timestamp of the speech corresponding to the character respectively; Determining a target time period corresponding to each character in the first text according to a start timestamp and an end timestamp of each character in the first text and a preset offset; Determine the editing range corresponding to each character in the first text in the second text according to the target time period corresponding to each character in the first text and the start timestamp and end timestamp of each character in the second text; the start timestamp and end timestamp of any character in the second text are the timestamp for starting to display the character and the timestamp for ending to display the character, respectively; The edit distance between each character in the first text and each character in an edit range corresponding to each character in the first text in the second text is calculated to obtain an edit distance between the first text and the second text.

5. The method according to claim 1, characterized in that The obtaining the edit distance between the first text and the second text includes: The first text and the second text are converted into pinyin respectively to obtain a first pinyin text and a second pinyin text; The edit distance between the first pinyin text and the second pinyin text is calculated to obtain the edit distance between the first text and the second text.

6. The method according to claim 5, characterized in that Before calculating the edit distance between the first phonetic text and the second phonetic text, the method further includes: The first pinyin text and the second pinyin text are fuzzified.

7. The method according to claim 1, characterized in that The step of obtaining a second text corresponding to the target speech according to the subtitles includes: Obtaining a playback time period of the target voice; Character recognition is performed on the display content of the subtitles within the play time period to obtain the second text.

8. The method according to claim 1, characterized in that The method further comprises: In a case where the edit distance between the first text and the second text is greater than the edit distance, the first text, the second text, and the target voice are discarded.

9. A data processing device, characterized in that: include: An acquisition unit, configured to acquire a target media file, wherein the target media file includes audio content and subtitles, and the audio content includes a target voice; A first recognition unit, configured to perform speech recognition on the target speech to obtain a first text; A second recognition unit, configured to obtain a second text corresponding to the target speech according to the subtitles; A processing unit, configured to obtain an edit distance between the first text and the second text; A generating unit is used to generate target data when the edit distance between the first text and the second text is less than or equal to a threshold edit distance, wherein the target data includes: the target speech and the target text, and the target text is generated based on the first text and / or the second text.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program and the processor is used to enable the electronic device to implement the data processing method according to any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the data processing method according to any one of claims 1 to 8.

Images