Method, device and equipment for correcting speech translation text and storage medium

Abstract

The application discloses a speech translation text error correction method and device, equipment and a storage medium. The method comprises the following steps: obtaining N pieces of translation text of a target audio, wherein N is an integer greater than 1; performing alignment processing on the N pieces of translation text, and determining a first error position of a first translation text based on the alignment processing result, wherein the first translation text is selected from the N pieces of translation text; performing error detection on the first translation text to obtain a second error position of the first translation text; and performing error correction processing on the first translation text based on the first error position and the second error position to obtain a corrected translation text of the target audio.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method, apparatus, device, and storage medium for error correction of speech-to-text translation. Background Technology

[0002] With the continuous development of artificial intelligence (AI) technology, speech recognition technology has been widely applied in the field of AI. Speech recognition technology converts human language into corresponding spoken text. Currently, the mainstream speech recognition technology is Automatic Speech Recognition (ASR). However, the spoken text transcribed by ASR often contains errors, significantly impacting AI's understanding of human conversations. Therefore, improving the accuracy of speech recognition is a crucial part of voice interaction. Correcting errors in the transcribed text is an effective method to improve speech recognition accuracy, requiring the provision of accurate text correction solutions. Summary of the Invention

[0003] The purpose of this application is to provide a method, apparatus, device, and storage medium for correcting errors in speech-to-text translation, so as to improve the accuracy of speech translation.

[0004] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] In a first aspect, embodiments of this application provide a method for correcting errors in speech-to-text translation, including:

[0006] Obtain N translated texts of the target audio, where N is an integer greater than 1;

[0007] The N translated texts are aligned, and the first error position of the first translated text is determined based on the alignment result; the first translated text is selected from the N translated texts.

[0008] Error detection is performed on the first translated text to obtain the second error location of the first translated text;

[0009] Based on the first error location and the second error location, the first translated text is corrected to obtain the corrected translated text of the target audio.

[0010] Secondly, embodiments of this application provide an error correction device for speech-to-text translation, comprising:

[0011] The acquisition module is used to acquire N transcribing texts of the target audio, where N is an integer greater than 1;

[0012] The determination module is used to align the N translated texts and determine the first error position of the first translated text based on the alignment result.

[0013] The determination module is further configured to perform error detection on the first translated text and obtain the second error location of the first translated text;

[0014] The correction module is used to perform error correction processing on the first translated text based on the first error position and the second error position to obtain the corrected translated text of the target audio.

[0015] Thirdly, embodiments of this application provide an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the speech-to-text error correction method as described in the first aspect.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium that, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the speech-to-text error correction method as described in the first aspect.

[0017] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0018] The audio transcription results were incorporated into the text correction process to assist in error correction. Since N transcribed texts were obtained from the same audio, aligning the N transcribed texts revealed the transcription uncertainties in the transcription process. For example, it revealed which position in the first transcribed text was uncertain in terms of transcription. This uncertainty was then used to pinpoint the possible erroneous character position in the first transcribed text, i.e., the first error position. Furthermore, error detection was performed on the first transcribed text to obtain the second error position. Since the error detection process might misidentify the erroneous transcription position in the first transcribed text, combining the first and second error positions for error correction could accurately correct errors in the transcribed text, improve the accuracy of speech transcription, and thus contribute to improving the accuracy of ASR transcription. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 A flowchart illustrating a speech-to-text error correction method provided in one embodiment of this application;

[0021] Figure 2 A flowchart illustrating a speech-to-text error correction method provided for another embodiment of this application;

[0022] Figure 3 A schematic diagram of the structure of a speech translation device provided in one embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0026] Explanation of some concepts:

[0027] ASR: Automatic Speech Recognition, is a technology that translates human speech into text.

[0028] ASR text correction: The text results translated by ASR are further corrected by an error correction model to improve the accuracy of ASR translated text.

[0029] Confidence: A score (between 0 and 1) on how well the translated text is translated in a certain scenario. The higher the score, the higher the confidence level.

[0030] This application, through research on ASR text correction techniques in related technologies, finds that most of these techniques train an ASR correction model based on historical translated text and the corrected text to correct errors in the ASR translated text. However, the ASR correction model has uncertainties regarding the translated text, which may lead to incorrect location of translation errors, resulting in incorrect correction and affecting the ASR performance.

[0031] Based on this, this application provides a method for correcting errors in speech-to-text translation. It uses N translated texts obtained through ASR translation and selects a first translated text from these N texts to obtain the translation uncertainty during the ASR translation process. This uncertainty is then used to determine the error location in the first translated text. Based on this, an error detection model is used to detect the error location in the first translated text, and the first translated text is then corrected. This method can accurately correct errors in the speech-to-text translation, improve the accuracy of the speech-to-text translation, and thus help improve the accuracy of ASR translation.

[0032] The speech-to-text error correction method provided in this application can be applied to various business scenarios with error correction requirements.

[0033] For example, in a financial quality inspection scenario, the audio of the conversation between the inspector and the customer service representative is transcribed into N transcribed texts. Based on the confidence level of the N transcribed texts, a first transcribed text is selected from the N transcribed texts. The first transcribed text is then corrected using the speech-transcribed text correction method provided in this application, combined with the N transcribed texts, to obtain a corrected transcribed text of the conversation audio. Finally, the service quality of the inspector is evaluated based on the corrected transcribed text to optimize the service quality.

[0034] For example, in a text input scenario, the audio input by the user is processed into speech-to-text translation to obtain N translated texts. Based on the confidence of the N translated texts, the first translated text is selected from the N translated texts. The speech-to-text error correction method provided in this application is used to correct the first translated text in combination with the N translated texts to obtain the corrected translated text of the audio. The corrected translated text is then displayed in the text input box.

[0035] For example, in an information retrieval scenario, the audio input by the user is processed into speech-to-text, resulting in N translated texts. Based on the confidence level of the N translated texts, a first translated text is selected from the N translated texts. The speech-to-text error correction method provided in this application is used to correct the first translated text in combination with the N translated texts to obtain a corrected translated text of the audio. Then, the information retrieval operation is performed based on the corrected translated text.

[0036] It should be understood that the speech-to-text error correction method provided in this application, applied to business scenarios such as financial quality inspection, text input, and information retrieval, is merely an illustrative example and should not be construed as limiting the application scenarios of the speech-to-text error correction method.

[0037] It should be understood that the speech-to-text error correction method provided in this application embodiment can be executed by an electronic device, specifically by the processor of the electronic device. The electronic device can be a terminal device, such as a smartphone, tablet computer, laptop computer, desktop computer, intelligent voice interaction device, smart home appliance, smartwatch, vehicle terminal, aircraft, etc.; or, the electronic device can also be a server, such as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0038] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0039] Please see Figure 1 The following is a flowchart illustrating a speech-to-text error correction method according to an embodiment of this application. The method may include the following steps:

[0040] S102, obtain N translated texts of the target audio.

[0041] N translated texts are carried in the speech translation result of the target audio. In practical applications, the speech translation result may also include the confidence level of each translated text.

[0042] In step S102 above, the target audio can be processed using various speech translation techniques to obtain the speech translation result. As an example, the speech translation result is obtained by processing the target audio using an ASR model.

[0043] It is worth noting that the value of N can be set according to actual needs, and this application embodiment does not limit it. For example, in a financial quality inspection scenario, N = 10.

[0044] S104, perform alignment processing on N translated texts, and determine the first error position of the first translated text based on the alignment processing result.

[0045] The first translated text is selected from N translated texts, for example, the translated text with the highest confidence level selected from the N translated texts. The N translated texts also include a second translated text. The second translated text is any one of the N translated texts excluding the first translated text, and the number of second translated texts is at least one. Unless otherwise specified, the following description of this application uses any one of the multiple second translated texts as an example. The same implementation method can be used for other second translated texts, and they will not be described in detail here.

[0046] The first error location is used to indicate the location in the first translated text where a translation error may exist.

[0047] Since the N translated texts are obtained from the same audio through speech translation, aligning the N translated texts can reflect the translation uncertainty in the speech translation process. For example, it can be uncertain which position in the first translated text will be translated. By using this uncertainty, the position of the character that may have a translation error in the first translated text can be accurately located, i.e., the first error position.

[0048] The alignment result includes character alignment information between the first and second translated texts. This character alignment information indicates the corresponding characters in the second translated text for each character in the first translated text.

[0049] In S104 above, the alignment of N translated texts can be implemented in various ways, and the specific method can be selected according to actual needs. This application embodiment does not limit this method.

[0050] In one embodiment, aligning N translated texts includes: Step A1, determining M alignment methods based on the difference in the number of characters between the first and second translated texts, where M is a positive integer; Step A2, aligning the characters of the first and second translated texts based on the M alignment methods to obtain M candidate character alignment information, which correspond one-to-one with the M alignment methods; wherein each candidate character alignment information includes character pairs corresponding to each position in the first translated text; the character pair corresponding to the i-th position includes the first character belonging to the i-th position in the first translated text and the second character belonging to the i-th position in the second translated text; the i-th position is any one of the positions; Step A3, selecting the candidate character alignment information with the most first character pairs from the M candidate character alignment information as the character alignment information between the first and second translated texts, wherein the first character and the second character contained in the first character pair are the same.

[0051] As an example, if the difference in the number of characters between the first and second translated texts is 0, then an alignment method is determined: align the i-th character of the first translated text with the i-th character of the second translated text, where i is a positive integer. If the difference in the number of characters between the first and second translated texts is greater than 1, then the determined alignment method can include at least one of the following: align the i-th character of the first translated text with the i-th character of the second translated text; align the i-th character of the first translated text with the (i+1)-th character of the second translated text, and so on.

[0052] For example, if the first translated text is "The weather is really nice today" and the second translated text is "The sky is really nice today", and the two texts have different lengths, aligning the characters of the two texts will yield at least two candidate character alignment options:

[0053] Candidate character alignment information 1:

[0054] {Jin-Jin: C, Tian-Tian: C, Tian-Tian: C, Qi-Zhen: S, Zhen-Hao: S, Hao-: D}

[0055] Candidate character alignment information 2:

[0056] {Jin-Jin: C, Tian-Tian: C, Tian-Tian: C, Qi-: D, Zhen-Zhen: C, Hao-Hao: C}

[0057] It can be seen that the candidate character alignment information 1 includes 6 character pairs: "Jin-Jin", "Tian-Tian", "Tian-Tian", "Qi-Zhen", "Zhen-Hao", and "Hao-". The candidate character alignment information 2 includes 6 character pairs: "Jin-Jin", "Tian-Tian", "Tian-Tian", "Qi-", "Zhen-Zhen", and "Hao-Hao". Wherein, the character on the left side of "-" is the first character, and the character on the right side of "-" is the second character, C indicates that the first character is identical to the second character, that is, a first character pair; D indicates that the first character is different from the second character, and one of the two is a null character; S indicates that the first character is different from the second character, and both of the two are non-null characters, that is, a second character pair.

[0058] By counting the character pairs in the candidate character alignment information 1, it is determined that the number of the first character pairs is 3. By counting the character pairs in the candidate character alignment information 2, it is determined that the number of the first character pairs is 5. Further, the candidate character alignment information 2 is determined as the character alignment information between the first transcribed text and the second transcribed text.

[0059] In this embodiment, considering that the lengths of the first transcribed text and the second transcribed text may be different, M pieces of candidate character alignment information are obtained by aligning the two character by character; since the first character and the second character in the first character pair are identical, the more first character pairs a piece of candidate character alignment information contains, the higher the reliability of the candidate character alignment information. Based on this, selecting the candidate character alignment information with the largest number of contained first character pairs can ensure the correctness of the character alignment information finally used for determining the error position, so as to more accurately reflect the transcription uncertainty in the speech transcription process, facilitate the precise correction of errors in the speech transcribed text, and improve the accuracy rate of speech transcription.

[0060] In another embodiment, N pieces of transcribed text may also be aligned by using an alignment algorithm commonly used in the art, for example, including but not limited to: a word number length-based alignment algorithm (Brown), a character number length-based alignment algorithm (Gale Church), and the like.

[0061] In the above S104, determining the first error position of the first translated text based on the alignment processing result may be implemented in various manners, which may be specifically selected according to actual needs, and the embodiments of the present application do not limit this.

[0062] In an implementation manner, determining the first error position of the first translated text based on the alignment processing result includes the following steps: Step B1: Determine candidate error positions in the first translated text based on character alignment information; Step B2: Count the number of target-type characters corresponding to the candidate error positions, wherein for each position, the target-type character refers to a character that is different from the character at that position in the first translated text; Step B3: Select the first error position of the first translated text from the candidate error positions based on the number of target-type characters corresponding to the candidate error positions.

[0063] In the above step B1, as an example, for each position, if the character at that position in the first translated text is different from the character at the same position in the second translated text, the position is determined as a candidate error position in the first translated text.

[0064] Taking N=3, the first translated text being "The weather is really nice today", and the second translated texts including "The weather is really nice today" and "The weather is really nice today" as an example, the character alignment information between the first translated text and these two second translated texts is as follows:

[0065] Character alignment information 1: {Jin-Jin: C, Tian-Tian: C, Tian-Tian: C, Qi-: D, Zhen-Zhen: C, Hao-Hao: C}

[0066] Character alignment information 2: {Jin-Jin: C, Tian-Tian: C, Tian-Tian: C, Qi-Shi: S, Zhen-Zhen: C, Hao-Hao: C}

[0067] In the above character alignment information, the first character "Qi" at the 4th position in the first translated text is different from the second character at the same position in at least one second translated text, so the 4th position is determined as a candidate error position in the first translated text. Further, it is determined that the characters at this position in the two second translated texts are both target-type characters, and the counted number of target-type characters corresponding to the 4th position is 2; similarly, the number of target-type characters corresponding to the remaining positions in the first translated text is 0.

[0068] As another example, based on the character alignment information, positions corresponding to second character pairs are selected from all positions of the first translated text, where the first character and the second character included in the second character pair are different, and both the first character and the second character are non-empty characters. In this case, in the above step B2, only the number of target-type characters at the positions corresponding to the second character pairs is counted.

[0069] Continuing with the example of character alignment information above, in character alignment information 2, the character pair corresponding to the 4th position in the first translated text is the second character pair, thus identifying the 4th position as a candidate error position. Furthermore, it is determined that the characters at this position in both second translated texts are target class characters, and the number of target class characters corresponding to the 4th position is counted as 2; similarly, the number of target class characters corresponding to the remaining positions in the first translated text is 0.

[0070] Understandably, in this example, only the number of target class characters at the position corresponding to the second character pair is counted. Based on the statistical results, the first error position is determined from this type of position to avoid the problem of inaccurate character change count caused by misalignment, thereby making a more accurate determination of the first error position.

[0071] In step B2 above, when there is at least one candidate error location, a candidate error location that meets the preset error correction conditions is selected from the candidate error locations and determined as the first error location of the first translated text. The number of target class characters corresponding to the candidate error locations that meet the preset error correction conditions meets the preset quantity requirement.

[0072] The preset quantity requirement can be set according to actual needs. For example, the number of the corresponding target class characters is greater than the number of the target class characters at other candidate error positions, or the number of the corresponding target class characters is greater than the preset quantity threshold, or the number of the corresponding target class characters is in the top 15% of all candidate error positions, etc. This application embodiment does not limit this.

[0073] For example, continuing with the above example, the preset quantity requirement is that the number of corresponding target class characters exceeds 1. Since the number of target class characters corresponding to the 4th position in the first translated text is 2, and the number of target class characters corresponding to the other positions is 0, the 4th position is designated as the first error position. In another embodiment, for each position in the first translated text, the first character at that position can be compared with its corresponding second character in the second translated text. If the first character at that position is not the same as the corresponding second character in at least one of the second translated texts, then that position is determined as the first error position.

[0074] S106, perform error detection on the first translated text to obtain the second error location of the first translated text.

[0075] Error detection of the first translated text can be performed using various error detection methods commonly used in the field, and this application embodiment does not limit this method. As an example, error detection of the first translated text can be performed based on an error detection model, wherein the error detection model is trained based on historical translated texts of historical audio and labeled error locations.

[0076] S108, based on the first error position and the second error position, perform error correction processing on the first transcribed text to obtain the corrected transcribed text of the target audio.

[0077] Because of the uncertainty in the translated text during the error detection process, the translation error position in the first translated text may be mislocated. However, the first error position determined by using N translated texts is more accurate. Combining the first error position and the second error position to perform error correction processing on the first translated text can accurately correct the errors in the speech-to-text, improve the accuracy of speech-to-text, and thus help improve the accuracy of ASR translation.

[0078] In one embodiment, S108 includes the following steps: based on the overlapping position of the first error position and the second error position, perform error correction processing on the characters of the first translated text at the overlapping position to obtain the corrected translated text.

[0079] Error correction processing for characters at overlapping positions can be implemented using various error correction methods commonly used in the art, and this application embodiment does not limit this. As an example, error detection of characters at overlapping positions in the first translated text can be performed based on an error correction model, wherein the error correction model is trained based on the error positions of historical translated texts and the correct characters at those error positions.

[0080] For example, the first error position includes the 4th and 5th positions of the first translated text, and the second error position includes the 5th and 6th positions of the first translated text. Then the 5th position is a common position. After the 5th position is marked in the first translated text, it is input into the error correction model. The error correction model corrects the characters at the 5th position and outputs the corrected translated text.

[0081] In this embodiment, the overlapping position of the first error position and the second error position is more likely to contain translation errors. By performing error correction processing on the characters in the overlapping position of the first translated text based on the error correction model, it is possible to avoid correcting characters that were originally translated correctly, reduce the error correction rate, and further improve the accuracy of speech translation, thereby helping to improve the accuracy of ASR translation.

[0082] In another embodiment, S108 includes the following steps: performing error correction processing on the characters at the first error position of the first translated text to obtain a first candidate corrected text; performing error correction processing on the characters at the second error position of the first translated text to obtain a second candidate corrected text; and determining the final corrected translated text based on the first candidate corrected text and the second candidate corrected text.

[0083] This application embodiment illustrates a partial implementation of S108 described above. It should be understood that S108 can also be implemented in other ways, and this application embodiment does not limit this implementation.

[0084] In another embodiment of this application, in order to avoid correcting the originally correct first translated text and reduce the error correction rate, and further improve the accuracy of speech translation, before the above S108, it is also determined whether the confidence level of the first translated text is less than a preset confidence level threshold; if so, the above S108 is executed; otherwise, the first translated text is output as the final result.

[0085] In practical applications, in order to reduce the consumption of computing resources, such as Figure 2 As shown, after S102 above, it is first determined whether the confidence level of the first translated text is less than the preset confidence level threshold; if so, S104 to S108 are executed, and the corrected translated text obtained through S108 is output as the final result; otherwise, the first translated text is output as the final result.

[0086] In addition, the confidence threshold can be set according to actual needs, and this application embodiment does not limit it.

[0087] In one implementation, the confidence threshold can be determined as follows:

[0088] Step C1: Obtain the historical speech translation results of K historical audio files.

[0089] The historical speech-to-text results include K reference translated texts and the confidence levels of the K reference translated texts. The K reference translated texts correspond one-to-one with the K historical audios. Each reference translated text is the translated text with the highest confidence level in the speech-to-text results of the corresponding historical audio. The K historical audios and the target audio come from the same business scenario, and K is an integer greater than 1.

[0090] In practical applications, the speech translation technology used for K historical audio samples is the same as that used for the target audio sample, such as using the same ASR model.

[0091] The specific implementation method of step C1 is the same as described above. Figure 1 The specific implementation of S102 in the illustrated embodiment is similar and will not be described again.

[0092] Step C2: Perform error detection on the K reference translated texts, and determine the reference translated text with translation errors from the K reference translated texts based on the error detection results.

[0093] As an example, K reference translated texts can be sent to the review platform for manual error detection to obtain reference translated texts with translation errors.

[0094] As another example, an error detection model can be used to detect errors in K reference translated texts; for each reference translated text, if the error position of the reference translated text is not empty, then it is determined that the reference translated text has a translation error.

[0095] Step C3: Determine the confidence threshold based on the confidence level of the reference translated text with translation errors.

[0096] By statistically analyzing the confidence level of the reference translated text with translation errors, it is possible to determine at what confidence level an error will occur. This threshold is then used to determine whether the first translated text needs to be corrected.

[0097] In this embodiment, by using the reference translated text and confidence level of historical audio in the same business scenario, the determined confidence level threshold can more accurately reflect the credibility of the first translated text in the business scenario, thereby avoiding the correction of the originally correct first translated text, reducing the error correction rate, and further improving the accuracy of speech translation.

[0098] One or more embodiments of this application provide a speech-to-text error correction method that incorporates the speech-to-text translation results of audio during the text error correction process. Since N translated texts are obtained from the same audio audio, aligning the N translated texts allows the alignment results to reflect the translation uncertainties in the speech-to-text process. For example, it may be uncertain which position in the first translated text will have a translation error. This uncertainty can then be used to accurately locate the character position in the first translated text that may have a translation error, i.e., the first error position. Furthermore, error detection is performed on the first translated text to obtain the second error position. Since the error detection process may incorrectly locate the translation error position in the first translated text, combining the first and second error positions to perform error correction processing on the first translated text can accurately correct errors in the speech-to-text, improve the accuracy of speech-to-text translation, and thus help improve the accuracy of ASR translation.

[0099] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0100] Based on the same inventive concept, embodiments of this application also provide a speech-to-text error correction device. Please see [link to relevant documentation]. Figure 3 The following is a schematic diagram of the structure of a speech-to-text error correction device 300 provided in an embodiment of this application. The device 300 includes: an acquisition module 310, a determination module 320 and a correction module 330.

[0101] The acquisition module 310 is used to acquire N transcribing texts of the target audio, where N is an integer greater than 1.

[0102] The determination module 320 is used to perform alignment processing on the N translated texts and determine the first error position of the first translated text based on the alignment processing result.

[0103] The determination module 320 is further configured to perform error detection on the first translated text to obtain the second error position of the first translated text.

[0104] The correction module 330 is used to perform error correction processing on the first translated text based on the first error position and the second error position to obtain the corrected translated text of the target audio.

[0105] In one embodiment, the N translated texts include a first translated text and a second translated text; the alignment result includes character alignment information between the first translated text and the second translated text; the number of the second translated texts is at least one.

[0106] When determining the first error position of the first translated text based on the alignment processing result, the determination module 320 performs the following steps:

[0107] Based on the character alignment information, the candidate error positions in the first translated text are determined;

[0108] Count the number of target characters corresponding to the candidate error positions; wherein, for each position, a target character refers to a character that is different from the character at that position in the first translated text;

[0109] Based on the number of target class characters corresponding to the candidate error positions, the first error position of the first translated text is selected from the candidate error positions.

[0110] In one embodiment, the determination module 320 aligns the N translated texts in the following manner:

[0111] Based on the difference in the number of characters between the first translated text and the second translated text, M alignment methods are determined, where M is a positive integer;

[0112] Based on the M alignment methods, the characters of the first translated text and the characters of the second translated text are aligned to obtain M candidate character alignment information, which correspond one-to-one with the M alignment methods. Each candidate character alignment information includes character pairs corresponding to various positions in the first translated text; the character pair corresponding to the i-th position includes the first character belonging to the i-th position in the first translated text and the second character belonging to the i-th position in the second translated text; the i-th position can be any one of the positions.

[0113] From the M types of candidate character alignment information, the candidate character alignment information that includes the most first character pairs is selected as the character alignment information between the first translated text and the second translated text, wherein the first character and the second character contained in the first character pair are the same.

[0114] In one embodiment, the determination module 320 determines the candidate error locations in the first translated text in the following manner:

[0115] Based on the character alignment information, the position corresponding to the second character pair is selected from each position of the first translated text, wherein the first character and the second character contained in the second character pair are different, and both the first character and the second character are non-empty characters;

[0116] The selected locations are determined as candidate error locations.

[0117] In one embodiment, the number of candidate error locations is at least one;

[0118] The determination module 320 selects the first error position of the first translated text from the candidate error positions in the following manner:

[0119] From the candidate error positions, the candidate error positions that meet the preset error correction conditions are selected and determined as the first error positions of the first translated text; wherein, the number of target class characters corresponding to the candidate error positions that meet the preset error correction conditions meets the preset number requirements.

[0120] In one embodiment, when the correction module 330 performs error correction processing on the first transcribed text based on the first error position and the second error position to obtain the corrected transcribed text of the target audio, it performs the following steps:

[0121] Determine the overlap position between the first error location and the second error location;

[0122] The first translated text is corrected by performing error correction processing on the characters at the overlapping positions based on the error correction model, resulting in the corrected translated text of the target audio.

[0123] In one embodiment, the N translated texts are carried in the speech translation result of the target audio, and the speech translation result also includes the confidence level of the first translated text;

[0124] When the correction module 330 performs error correction processing on the first translated text based on the first error position and the second error position to obtain the corrected translated text of the target audio, it performs the following steps:

[0125] If the confidence level of the first translated text is less than a preset confidence level threshold, then based on the first error location and the second error location, the first translated text is corrected to obtain the corrected translated text of the target audio.

[0126] In one embodiment, the acquisition module 310 is further configured to:

[0127] Before the correction module 330 performs error correction processing on the first translated text based on the first error position and the second error position to obtain the corrected translated text of the target audio, it obtains the historical speech translation results of K historical audios. The K reference translated texts correspond one-to-one with the K historical audios. The historical speech translation results include the K reference translated texts and the confidence level of the K reference translated texts. The K historical audios and the target audio come from the same business scenario, and K is an integer greater than 1.

[0128] The determination module 320 is further configured to: perform error detection on the K reference translated texts, and determine the reference translated text with translation error from the K reference translated texts based on the error detection results;

[0129] The device 300 further includes:

[0130] A determining unit is used to determine the confidence threshold based on the confidence level of the reference translated text with the translation error.

[0131] Obviously, the speech-to-text error correction device provided in this application embodiment can serve as... Figure 1 The entity executing the speech-to-text error correction method shown is, for example... Figure 1 In the speech-to-text error correction method shown, step S102 can be performed by... Figure 3 The acquisition module 310 in the speech-to-text error correction device shown executes steps S104 and S106, which can be performed by... Figure 3 The judgment module 320 in the speech-to-text error correction device shown executes step S108, which can be performed by... Figure 3 The correction module 330 in the speech-to-text error correction device shown is executed.

[0132] According to another embodiment of this application, Figure 3 The various modules in the speech-to-text error correction device shown can be individually or entirely merged into one or more other modules, or some of the modules can be further divided into multiple functionally smaller modules. This achieves the same operation without affecting the technical effect of the embodiments of this application. The above modules are based on logical function division. In practical applications, the function of one module can be implemented by multiple modules, or the function of multiple modules can be implemented by one module. In the embodiments of this application, the speech-to-text error correction device may also include other modules. In practical applications, these functions can also be implemented with the assistance of other modules, and can be implemented collaboratively by multiple modules.

[0133] According to another embodiment of this application, a general-purpose computing device, such as a computer, including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can run an application capable of performing tasks such as... Figure 1 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 3 The document describes a speech-to-text error correction device and a speech-to-text error correction method for implementing embodiments of this application. The computer program may be recorded on, for example, a computer-readable storage medium, and may be transferred to and run in an electronic device via such a medium.

[0134] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 4At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0135] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0136] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0137] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a speech-to-text error correction device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0138] Obtain N translated texts of the target audio, where N is an integer greater than 1;

[0139] The N translated texts are aligned, and the first error position of the first translated text is determined based on the alignment result; the first translated text is selected from the N translated texts.

[0140] Error detection is performed on the first translated text to obtain the second error location of the first translated text;

[0141] Based on the first error location and the second error location, the first translated text is corrected to obtain the corrected translated text of the target audio.

[0142] The above is as stated in this application. Figure 1The method executed by the speech-to-text error correction device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0143] The electronic device can also perform Figure 1 The method, and the implementation of an error correction device for speech-to-text translation. Figure 1 , Figure 2 The functions of the embodiments shown are not described in detail here.

[0144] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0145] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment is specifically used to perform the following operations:

[0146] Obtain N translated texts of the target audio, where N is an integer greater than 1;

[0147] The N translated texts are aligned, and the first error position of the first translated text is determined based on the alignment result; the first translated text is selected from the N translated texts.

[0148] Error detection is performed on the first translated text to obtain the second error location of the first translated text;

[0149] Based on the first error location and the second error location, the first translated text is corrected to obtain the corrected translated text of the target audio.

[0150] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0151] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0152] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0153] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0154] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A method for error correction in speech-to-text translation, characterized in that, include: Obtain N translated texts of the target audio, where N is an integer greater than 1; The N translated texts are aligned, and the first error position of the first translated text is determined based on the alignment result. The first translated text is selected from the N translated texts; Error detection is performed on the first translated text to obtain the second error location of the first translated text; Based on the first error location and the second error location, the first transliterated text is subjected to error correction processing to obtain the corrected transliterated text of the target audio; wherein, the error correction processing includes: determining the overlapping position of the first error location and the second error location; and performing error correction processing on the characters of the first transliterated text at the overlapping position to obtain the corrected transliterated text of the target audio.

2. The method according to claim 1, characterized in that, The N translated texts include a first translated text and a second translated text; the alignment result includes character alignment information between the first translated text and the second translated text; The number of the second translated text is at least one; The determination of the first error location in the first translated text based on the alignment processing result includes: Based on the character alignment information, the candidate error positions in the first translated text are determined; The number of target class characters corresponding to the candidate error positions is counted; wherein, for each candidate error position, the target class character refers to a character that is different from the character at the candidate error position in the first translated text; Based on the number of target class characters corresponding to the candidate error positions, the first error position of the first translated text is selected from the candidate error positions.

3. The method according to claim 2, characterized in that, The alignment process for the N translated texts includes: Based on the difference in the number of characters between the first translated text and the second translated text, M alignment methods are determined, where M is a positive integer; Based on the M alignment methods, the characters of the first translated text and the characters of the second translated text are aligned to obtain M candidate character alignment information, which correspond one-to-one with the M alignment methods. Each candidate character alignment information includes character pairs corresponding to various positions in the first translated text; the character pair corresponding to the i-th position includes the first character belonging to the i-th position in the first translated text and the second character belonging to the i-th position in the second translated text; the i-th position can be any one of the positions. From the M types of candidate character alignment information, the candidate character alignment information that includes the most first character pairs is selected as the character alignment information between the first translated text and the second translated text, wherein the first character and the second character contained in the first character pair are the same.

4. The method according to claim 3, characterized in that, Determining the candidate error positions in the first translated text based on the character alignment information includes: Based on the character alignment information, the position corresponding to the second character pair is selected from each position of the first translated text, wherein the first character and the second character contained in the second character pair are different, and both the first character and the second character are non-empty characters; The selected locations are determined as candidate error locations.

5. The method according to claim 4, characterized in that, The number of candidate error locations is at least one; the step of selecting the first error location of the first translated text from the candidate error locations based on the number of target class characters corresponding to the candidate error locations includes: From the candidate error positions, the candidate error positions that meet the preset error correction conditions are selected and determined as the first error positions of the first translated text; wherein, the number of target class characters corresponding to the candidate error positions that meet the preset error correction conditions meets the preset number requirements.

6. The method according to claim 1, characterized in that, The N translated texts are carried in the speech translation result of the target audio, and the speech translation result also includes the confidence level of the first translated text; The step of correcting the first translated text based on the first error location and the second error location to obtain the corrected translated text of the target audio includes: If the confidence level of the first translated text is less than a preset confidence level threshold, then the first translated text is corrected based on the first error location and the second error location to obtain the corrected translated text.

7. The method according to claim 6, characterized in that, Before performing error correction processing on the first translated text based on the first error location and the second error location to obtain the corrected translated text of the target audio, the method further includes: Obtain the historical speech translation results of K historical audios. The historical speech translation results include K reference translated texts and the confidence scores of the K reference translated texts. The K reference translated texts correspond one-to-one with the K historical audios. The K historical audios and the target audios originate from the same business scenario. K is an integer greater than 1. Error detection is performed on the K reference translated texts, and the reference translated text with translation errors is determined from the K reference translated texts based on the error detection results; The confidence threshold is determined based on the confidence level of the reference translated text with the translation error.

8. A speech-to-text error correction device, characterized in that, include: The acquisition module is used to acquire N transcribing texts of the target audio, where N is an integer greater than 1; The determination module is used to align the N translated texts and determine the first error position of the first translated text based on the alignment result; the first translated text is selected from the N translated texts. The determination module is further configured to perform error detection on the first translated text and obtain the second error location of the first translated text; The correction module is used to perform error correction processing on the first transcribed text based on the first error position and the second error position to obtain the corrected transcribed text of the target audio; wherein the error correction processing includes: determining the overlapping position of the first error position and the second error position; and performing error correction processing on the characters of the first transcribed text at the overlapping position to obtain the corrected transcribed text of the target audio.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the error correction method for speech-to-text translation as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the error correction method for speech-to-text as described in any one of claims 1 to 7.

Images