Speech translation method, device, equipment and storage medium

Abstract

The embodiments of the present application provide a speech translation method, apparatus, device and storage medium. The method recognizes and judges the input speech signal, accumulates at least one of the literal elements to form a judgment unit, obtains the literal unit based on the BERT classification model and the judgment unit, and then segments the literal unit to form a target word segment; finally, based on the Transformer translation model, the target word segment is translated and output. The use of this solution greatly shortens the length of the sentence to be translated, avoids translation problems caused by complex semantics and sentence components in long sentences, and improves the accuracy of the translation; at the same time, the input of speech, the formation of the target word segment and the translation can be carried out simultaneously, which improves the translation speed.

Description

Technical Field

[0001] The embodiments of the present application relate to but are not limited to the field of artificial intelligence, and in particular, to a speech translation method, apparatus, device, and computer-readable storage medium. Background Art

[0002] Machine translation is a technology that uses a program to translate one natural language into another. This technology has been widely used in all walks of life. Machine translation has evolved from early rule-based technology to the current use of neural network technology. At present, machine translation works well in written translation and basically meets the requirements of high accuracy and low latency. However, when it is applied to voice translation such as voice interpretation, the effect is not good, and the translation accuracy and latency need to be improved. Summary of the invention

[0003] The embodiments of the present application provide a speech translation method, apparatus, device and computer-readable storage medium, which directly segment the input speech signal into source text and translate it synchronously, thereby achieving the effect of shortening the delay.

[0004] In the first aspect, the embodiment of the present application provides a speech translation method, including: obtaining a source speech signal, identifying the literal elements in the source speech signal; sequentially accumulating at least one of the literal elements to form a judgment unit; based on the BERT classification model, calculating the probability value of the judgment unit under the preset classification result condition; if the probability value is greater than the first threshold, the judgment unit is a literal unit; segmenting the literal unit to form a target word segment; based on the Transformer translation model, translating and outputting the target word segment. Wherein, the literal element is at least any one of the following: a single word, a word, a phrase or an idiom. The method provided in this embodiment directly obtains and accumulates the literal elements identified in the speech signal to be translated, such as words, and when the accumulated words meet the requirements of constituting a word segment with a complete and clear interpretation, the word segment is obtained as a literal unit, and the literal unit is segmented and translated and output, which greatly simplifies the translation process, effectively reduces the time required for translation, and shortens the delay; and the three processes of obtaining the speech signal, obtaining the literal unit segmentation and translating are carried out simultaneously, further shortening the delay.

[0005] In a second aspect, an embodiment of the present application provides a speech translation device, characterized in that it includes: a speech recognition module, used to obtain a source speech signal and identify the literal elements in the source speech signal; a literal unit acquisition module, used to sequentially accumulate at least one of the literal elements to form a judgment unit, based on the BERT classification model, calculate the probability value of the judgment unit under a preset classification result condition, and select the judgment unit with a probability value greater than a first threshold as the literal unit; a segmentation module, used to segment the literal unit to form a target word segment; a translation module, used to receive the target word segment, translate and output the literal element as at least any one of the following: a single word, a word, a phrase or an idiom.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes a method for executing the first method.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that it stores program instructions executable by a processor, wherein the program instructions are used to execute the first method.

[0008] The embodiment of the present application recognizes and judges the input voice signal, accumulates at least one of the literal elements to form a judgment unit, obtains the literal unit based on the BERT classification model and the judgment unit, and then segments the literal unit to form a target word segment; finally, based on the Transformer translation model, the target word segment is translated and output. The use of this solution greatly shortens the length of the sentence to be translated, avoids translation problems caused by complex semantics and sentence components in long sentences, and improves the accuracy of translation; at the same time, the input of voice, the formation of target word segments and translation can be carried out simultaneously, which improves the translation speed.

[0009] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A flowchart of a speech translation method provided in an embodiment of the present application;

[0011] Figure 2 A flowchart of a process for obtaining a meaningful unit according to an embodiment of the present application;

[0012] Figure 3This is a schematic diagram of the classification results of the judgment unit based on the BERT model involved in the embodiment of the present application;

[0013] Figure 4 A schematic diagram of a BERT model for probability judgment of a meaningful unit involved in an embodiment of the present application;

[0014] Figure 5 It is a schematic diagram of the flow chart of the segmentation of the real unit involved in the embodiment of the present application;

[0015] Figure 6 It is a schematic diagram of the subdivision of the real unit involved in the embodiment of the present application;

[0016] Figure 7-9 A schematic diagram of a model of a subdivision step of a real unit involved in an embodiment of the present application;

[0017] Fig.10 This is a schematic diagram of the synchronous operation of the speech translation method according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined arbitrarily with each other.

[0019] The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that if it involves a description of an orientation, such as up, down, front, back, left, right, etc., the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0020] It should be noted that at least one means one or more, more means more than two, greater than, less than, and exceeding are understood as excluding the number itself, and above, below, and within are understood as including the number itself. If there is a description of first or second, it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0021] With the further development of translation technology and the continuous changes in people's needs, the current research direction of machine translation has gradually shifted from written translation to voice translation. At present, voice translation technology works well in short sentences, and the translation accuracy and latency can basically meet the needs. However, when applied to simultaneous interpretation, due to the increase in sentence length, the increase in important information contained in the sentence, and the increase in the complexity of the sentence components, the accuracy of voice translation decreases and the latency increases.

[0022] In this regard, the present application proposes a speech translation method, which at least includes:

[0023] A source speech signal is obtained, and the meaningful elements in the source speech signal are identified.

[0024] In some embodiments, the source voice signal is the voice signal to be translated; the source voice signal can be a common language such as Chinese, English, French, Spanish, Russian, etc., or can be other languages ​​such as Cantonese, Japanese, Polish, Indonesian, etc.

[0025] In some embodiments, the content element may be: a single word, character, word, phrase or idiom in a sentence.

[0026] At least one literal element is sequentially accumulated to form a judgment unit.

[0027] In some embodiments, these judgment units are divided into two categories after judgment, namely, literal units and non-literal units. The literal units can be segmented and then translated and output, while non-literal units need to be further accumulated to eventually form literal units. The literal unit is a word segment or short sentence with a complete and clear interpretation; that is, the literal unit is a word segment or short sentence with a complete and clear interpretation composed of sentence information such as "single words, characters, words, phrases or idioms". By forming literal units, the accuracy of speech translation can be greatly improved.

[0028] In other embodiments, the meaning of the literal unit may not be affected by the literal elements subsequently recognized, and may not affect the meaning of other literal units. Since the meanings of each literal unit do not affect each other, and the subsequently acquired speech signals do not affect the meanings of the literal units, not only the accuracy of the translation can be improved, but also the translation speed can be increased.

[0029] Based on the BERT classification model, the probability value of the judgment unit under the preset classification result conditions is calculated.

[0030] In some embodiments, the full name of the BERT model is Bidirectional Encoder Representations from Transformer, which is a bidirectional encoder representation based on Transformer. BERT uses the Transformer translation model, and when processing a semantic element, it can also consider the words before and after the semantic element to get its meaning in the context. Therefore, the Transformer's attention mechanism has a good effect in extracting features from words in context, and intuitively, bidirectional encoding that considers context is better than unidirectional encoding that only considers the previous (or following) context.

[0031] In some embodiments, the judgment unit is input into the BERT model to obtain the conditional probability under the preset classification result. By setting a preset threshold and comparing the calculated conditional probability with the threshold, the final classification result can be obtained.

[0032] If the probability value is greater than a preset first threshold, the unit is determined to be a literal unit.

[0033] In some embodiments, a value greater than the first threshold is a meaningful unit; correspondingly, a value less than the first threshold is a non-meaningful unit.

[0034] Segment the meaning units to form target segments;

[0035] In some embodiments, the acquisition and optimization of the target word segment depends on the attention mechanism in the Transformer translation model. When the translation result is not ideal, the segmentation method can be adjusted in reverse or the meaningful unit can be expanded.

[0036] Based on the Transformer translation model, the target segment is translated and output.

[0037] In some embodiments, the output may be a translated text signal or a voice signal.

[0038] This embodiment recognizes and judges the input voice signal, obtains the literal unit based on the BERT classification model, and then translates the literal unit after segmentation, which is conducive to improving the accuracy of translation. In addition, the input of the voice signal, the judgment and formation of the literal unit, and the translation of the word segment or short sentence can be carried out simultaneously in parallel to reduce the translation delay. At the same time, since the interpretation of the literal unit is not affected by the subsequent input voice, the translation delay can be reduced in advance while ensuring the accuracy of the translation to meet the needs of machine simultaneous interpretation.

[0039] Figure 1This is a flowchart of a speech translation method provided by another embodiment of the present application. This embodiment uses words as literal elements to further illustrate the technical solution of the present application. Figure 1 The speech translation method provided in the embodiment at least includes:

[0040] Step S100: Acquire a source speech signal and identify meaningful elements in the source speech signal.

[0041] In some embodiments, the source speech signal may be acquired and recognized by a speech recognition (Auto Speech Recognition, ASR) system.

[0042] In some embodiments, the source voice signal is the voice signal to be translated; the source voice signal can be a common language such as Chinese, English, French, Spanish, Russian, etc., or can be other languages ​​such as Cantonese, Japanese, Polish, Indonesian, etc.

[0043] In some embodiments, the content element may be: a single word, character, word, phrase or idiom in a sentence.

[0044] Step S200: sequentially accumulating at least one literal element to form a judgment unit.

[0045] In some embodiments, at least one word input one by one is accumulated sequentially to form a judgment unit.

[0046] In some embodiments, accumulation is performed one by one, that is, the judgment unit formed by the accumulation of each literal element will be input into the model for classification judgment.

[0047] In some embodiments, accumulation may be performed group by group, that is, multiple literal elements are accumulated each time, which can improve the speed and efficiency of translation.

[0048] It is worth noting that the meaning of sequential accumulation is to accumulate the semantic elements in sequence according to the received source speech signal. Its essential purpose is to obtain judgment units that conform to the word order so that the subsequent translation can be more consistent with the meaning expressed by the source speech.

[0049] Step S300: Based on the BERT classification model, the probability value of the judgment unit under the preset classification result condition is calculated.

[0050] The BERT classification model can classify the input judgment unit and output its classification results.

[0051] In some embodiments, when the classification result is equal to 1, it indicates that the judgment unit is a literal unit, and when the classification result is equal to 0, it indicates that the judgment unit is a non-literal unit. Through the calculation of the BERT model, the conditional probability under the preset classification result can be output.

[0052] Since the BERT model needs to improve the accuracy of classification through comprehensive calculation of context, a preparatory unit is defined in the embodiment of the present application. The preparatory unit is composed of literal elements that are sequentially accumulated but not entered into the judgment unit, representing the semantic content of the context. Based on the BERT classification model, both the judgment unit and the preparatory unit are input into the model, and the conditional probability of the classification result of the judgment unit is output after calculation.

[0053] In some embodiments, if the preset classification result is a meaningful unit, that is, the output result is 1, then the calculated conditional probability value is the probability that the classification result is a meaningful unit.

[0054] In some embodiments, the classification result may be preset as a non-literal unit, that is, the output result is 0, and then the calculated conditional probability value is the probability that the classification result is a non-literal unit.

[0055] The specific classification result of the judgment unit is determined by the output conditional probability and the preset first threshold.

[0056] Step S400: If the probability value is greater than a preset first threshold, the unit is determined to be a literal unit.

[0057] The setting of the first threshold is related to the final classification result. The higher the preset threshold, the less likely it is that the judgment unit will be classified as a meaningful unit. At this time, the accuracy of the model output will be higher, but the translation efficiency will be affected. Therefore, the selection of the first threshold often requires calculation during the training process to obtain a suitable threshold.

[0058] In some embodiments, if the preset classification result is a literal unit and the first threshold is 0.7, then when the output result of the model, i.e., the conditional probability, is greater than 0.7, the judgment unit is a literal unit, and the judgment unit can be segmented to participate in the next calculation.

[0059] In some embodiments, if the preset classification result is a non-literal unit and the first threshold is 0.7, then when the output result of the model, i.e., the conditional probability, is greater than 0.7, the judgment unit is a non-literal unit, and the judgment unit needs to be reconstructed and the calculation repeated.

[0060] Step S500: segmenting the lexical units to form target segments.

[0061] The acquisition and optimization of the target word segment depends on the attention mechanism in the Transformer translation model. When the translation result is not ideal, the segmentation method can be adjusted in reverse or the semantic unit can be expanded.

[0062] In some embodiments, the attention value of each literal element in the current literal unit is obtained, and the position where the literal element with the largest attention value appears is determined, wherein the attention value is obtained based on the attention mechanism model of the Transformer translation model. If the literal element with the largest attention value appears at the preset position of the literal unit, it means that the current segmentation rule meets the requirements, and the literal unit is segmented to form a target segment.

[0063] In some embodiments, the attention value of each literal element in the current literal unit is also obtained, and the position where the literal element with the largest attention value appears is determined, wherein the attention value is obtained based on the attention mechanism model of the Transformer translation model. If the literal element with the largest attention value does not appear at the preset position of the literal unit, it means that the current segmentation rule does not meet the requirements, and the current literal unit is expanded. The attention value of each literal element in the expanded literal unit is obtained again, and the position where the word with the largest attention value appears is determined, until the literal element with the largest attention value appears at the preset position of the literal unit, indicating that the requirements are met, then the expansion is stopped, and the literal unit composed of the final expansion is segmented to form the target segment.

[0064] In some embodiments, the literal unit may not be expanded through the attention mechanism. Instead, when it is determined that the translation effect is not good enough, the current literal unit is directly expanded until a new literal unit is formed. The essential purpose is to increase the number of literal elements in the literal unit.

[0065] Step S600: Based on the Transformer translation model, the target segment is translated and output.

[0066] In some embodiments, the target segment may be translated instantly by a machine translation (MT) system.

[0067] Those skilled in the art know that, since the length of the target word segment is less than or equal to the semantic unit, translating the target word segment can be more accurate and, due to its synchronization, can also improve translation efficiency.

[0068] This embodiment accumulates the inputted literal elements one by one, and when the accumulated literal elements form a literal unit, the current word is segmented to form a target word segment, and then the target word segment is translated. On the one hand, the length of the sentence to be translated is greatly shortened, the translation problems caused by the complex sentence components in the long sentence are avoided, and the accuracy of the translation is improved; on the other hand, the input of speech, the formation of literal units, the formation of target word segments and the translation can be carried out simultaneously, which improves the translation speed and efficiency.

[0069] It should be noted that steps S100 to S500 can be performed synchronously at the same time. When the translation module is translating the first target segment corresponding to the first literal unit, the segmentation module is also segmenting the known topic units. At the same time, the literal unit acquisition module is accumulating newly input literal elements to form a literal unit with a complete and clear interpretation, that is, forming a second literal unit. At the same time, the speech recognition module is also continuously receiving the words spoken by the translator. That is, steps S100 to S500 work simultaneously and in parallel. The speech translation method provided in the embodiment of the present application can reduce the delay while ensuring the accuracy of the translation.

[0070] Figure 2 The following is a flow chart of a process for obtaining a meaningful unit according to an embodiment of the present application. The specific steps are shown in the figure.

[0071] In the embodiment of the present application, the literal element is a word.

[0072] Step S211: sequentially accumulate at least one word input one by one.

[0073] In the embodiment of the present application, the literal elements are added one by one to form a judgment unit.

[0074] Step S212, calculating the probability value under the preset classification result condition.

[0075] In some embodiments, the preset classification result is 1, that is, a literal unit, then the calculated probability value is the probability value that the unit composed of the current word is a literal unit, which can be directly compared with the threshold.

[0076] In some embodiments, the preset classification result is 0, that is, a non-literal unit, then the calculated probability value is the probability value that the unit composed of the current word is a non-literal unit, and the probability value obtained by subtracting the probability value from 1 is used to compare with the threshold.

[0077] Step S213, determining whether the probability value is greater than a preset threshold.

[0078] In the embodiment of the present application, the preset classification result is 1, that is, a literal unit, so the conditional probability that the judgment result is a literal unit is calculated, and the preset threshold is 0.7.

[0079] Step S214: if the probability value is greater than the threshold, the judgment unit formed by the current word is a meaningful unit.

[0080] Step S215: If the probability value is less than or equal to the threshold, continue to accumulate at least one word and then return to step S212.

[0081] More specifically, in some embodiments, if the probability value is less than or equal to a threshold value, it means that the word segment A1 composed of the words accumulated so far does not meet the requirements of being a meaningful unit, and then it is necessary to continue to accumulate and add at least one word on the basis of the current word segment A1 to form a new word segment A2, and then recalculate the probability value of the classification result of the new word segment A2 to determine whether the probability value is greater than the threshold value; if the probability value of the new word segment A2 is still less than or equal to the threshold value, it is necessary to continue to accumulate and add at least one word to form new words segments A3, A4, A5...An, until the probability value of the classification result of the word segment An is greater than the threshold value, then the word segment An is determined to be a meaningful unit, where n is a positive integer.

[0082] In some embodiments, the classification of the meaningful units in step S210 is performed using a BERT model. Figure 3 and attached Figure 4 For further explanation.

[0083] For example, if we need to translate the Chinese pronunciation “shangwu 10 dian wo qule tang gong yuan” into the English pronunciation “At 10 am l went to the part”, t is used to represent the number of input words; ct = {x ≤t} is used to represent the entry word; f t ={x t+1 , ..., x t+m} is used to represent the non-entered words; m is used to represent the number of non-entered words. In this embodiment, m is set to 2, and the value of m can be set according to actual conditions; p(l t =1|c t , f t θ model ) represents c t is the classification result l t The probability of being 1 (class 1) is set to the threshold δ=0.7.

[0084] When t = 1, c t ={shangwu},f t (m=2)={10 dian}, assuming that the calculated probability value is 0.1, 0.1 is less than the threshold value 0.7, then the classification result is 0, indicating that the current unit is not a literal unit and more words need to be accumulated.

[0085] In order to ensure the accuracy of translation, the present embodiment sets the accumulation to be increased word by word. Of course, in other embodiments, more words may be added as needed.

[0086] When t = 2, c t ={shangwu 10},f t(m=2)={dian wo}, assuming that the calculated probability value is 0.4, 0.4 is less than the threshold value 0.7, then the classification result is 0, indicating that the current unit is not a literal unit and more words need to be accumulated.

[0087] When t = 3, c t ={shangwu 10 dian}, f t (m=2)={wo qule}, assuming that the calculated probability value is 0.9, 0.9 is greater than the threshold value 0.7, then the classification result is 1, indicating that the segment "shangwu 10 dian" meets the standard of being a meaningful unit, and then it is determined that the segment "shangwu 10 dian" formed by the accumulated words is a meaningful unit.

[0088] The judgment method for t=4, 5, 9... is similar to that for t=1, 2, 3, and will not be explained one by one here.

[0089] Figure 5 A schematic diagram of a flow chart of a semantic unit segmentation based on an attention mechanism of a Transformer translation model provided in an embodiment of the present application, according to Figure 5 The embodiment of the present application shown includes at least the following steps:

[0090] Step S511: Obtain the attention value of each literal element in the current literal unit.

[0091] The attention value can reflect the impact of each semantic element on the translation result during the translation process, and reflects the importance of each semantic element. Therefore, in the model, important semantic elements usually have larger weight values, and unimportant semantic elements usually have smaller weight values. Therefore, important semantic elements can be pre-set by the model trainer, or they can be continuously formed and corrected through the training process.

[0092] In some embodiments of the present application, the conventional attention mechanism of the Transformer model is used to calculate the attention value, that is, the weight, of each meaningful element.

[0093] In some embodiments, the attention value a of each word in the meaning unit is first calculated by the first formula: jt ,

[0094]

[0095] Then calculate the maximum attention value a through the second formula j ,

[0096]

[0097] Wherein, t is used to characterize XXX, and 1≤t<g(j), t′ is used to characterize XXX, j is used to characterize XXX, and g(j) is used to characterize XXX.

[0098] Step S512: Obtain the literal element with the maximum attention value.

[0099] In the embodiment of the present application, the meaningful element with the maximum attention value is obtained by comparing the weights.

[0100] Step S513: Obtain the position where the meaningful element with the largest attention value appears.

[0101] In the embodiment of the present application, the position where the literal element appears can be divided into the beginning, the middle or the end of the literal unit.

[0102] Step S514: Determine whether the word with the largest attention value appears at the preset position of the meaning unit.

[0103] In some embodiments of the present application, if the preset position is the middle, and the position of the literal element with the largest calculated attention value is the middle, it means that the segmentation rule can contain important literal elements and also contains the context of the important literal elements, which is one of the best situations. Therefore, the literal unit can be segmented according to the segmentation rule at this time. Then, step S515 is performed.

[0104] In some embodiments of the present application, if the preset position is the middle, and the position where the semantic element with the largest calculated attention value appears is the end, it means that although this segmentation rule contains important semantic elements, it does not contain the following text of the important semantic elements, which may lead to semantic loss and does not meet the requirements. Therefore, it is necessary to enter step S516 to expand the semantic unit, so that the expanded semantic unit can meet the current segmentation rule.

[0105] In some embodiments, the expansion of the semantic unit is to add at least one word to the phrase corresponding to the previous semantic unit. Each expansion may be to add one word, or two, or more words, and then return to step S221 to calculate the attention value for each word of the newly expanded semantic unit.

[0106] Step S515: forming a target word segment.

[0107] Step S516: Expand the current literal unit.

[0108] It should be noted that it is also possible to expand the semantic unit without using the attention mechanism. Instead, when it is judged that the translation effect is not good enough, the current semantic unit is directly expanded until a new semantic unit is formed. Its essential purpose is to increase the number of semantic elements in the semantic unit.

[0109] Figure 6 It is a specific example of a semantic unit segmentation, as shown in the figure.

[0110] To more clearly illustrate the method for judging the segmentation position, the following combines the specific Chinese-English sentence "I watch TV while you eat" and the appendix Figure 6 for further analysis.

[0111] Define the source text as x = {x1, x2,..., x T}, and define g(j) as a monotonically non-decreasing function, which represents the position from the start of the encoder to the current position. During the decoding process, only the sequence x ≤g(j) = {x1, x2,..., x g(j)} can be determined as the target word segment y i , and 0 < g(j) ≤ T, where T is a positive integer used to represent the word number.

[0112] Determining g(j) first is very important for how to determine each target word segment yi. In the appendix Fig. 9 , the source text source x = {"I", "watch", "TV", "you", "eat"}. When the decoding process j = 2, it means that the segmentation model needs to use x ≤2 to generate the target word segment y2. However, in this case, the segmentation model makes a mistake because the insufficient context causes ambiguity in the translation process, resulting in the segmentation model being unable to determine whether "watch" should be translated as "look", "watch", or "see". Therefore, when setting g(j), sufficient context should be given for translation. In the understanding of segmentation, since in the previous steps, the required semantic units have been generated, and the semantic unit in this embodiment is "I watch", it is necessary to determine whether this word segment should be segmented at this semantic unit. As recorded above, for the initially judged semantic unit "I watch", during the actual translation process, there is ambiguity and an error occurs in the translation of the word "watch". This shows that in the initial division of the semantic unit "I watch", there is a translation risk for the semantic unit under this division, and more sufficient context needs to be provided to support accurate translation. Therefore, the semantic unit needs to be extended.

[0113] In some embodiments, in order to identify the risks existing in the above translation, during segmentation, the attention mechanism a jt is used to calculate the probability of the source text x t (1 ≤ t < g(j)) and the target word segment y j .

[0114] First, calculate the attention value a jt of each word in the semantic unit through the first formula,

[0115]

[0116] Then, calculate the maximum attention value a through the second formula j ,

[0117]

[0118] where t is used to represent XXX, and 1 ≤ t < g(j), t′ is used to represent XXX, j is used to represent XXX, and g(j) is used to represent XXX.

[0119] In the appendix Figure 6 appears the word "look" with the largest attention value in the source text, and it appears at the end of the pre-extraction block in the semantic unit "I look". This situation will cause the current module to be responsible for not being able to effectively predict the subsequent content during the translation process. In this case, it is necessary to expand the semantic unit to reduce the problem of insufficient content required for translation.

[0120] In some embodiments, the expansion of the semantic unit is to add at least one word based on the phrase corresponding to the previous semantic unit. Each expansion can be adding one word, two words, or multiple words, and then recalculate the attention value for each word of the newly expanded semantic unit.

[0121] Figure 7 This is the voice translation device provided by an embodiment of the present application, as Figure 7 shown.

[0122] The voice translation device in the embodiments of the present application at least includes: a voice recognition module for acquiring a source voice signal and identifying semantic elements in the source voice signal; a semantic unit acquisition module for sequentially accumulating at least one of the semantic elements to form a judgment unit, calculating the probability value of the judgment unit under a preset classification result condition based on a bert classification model, and selecting the judgment unit with a probability value greater than a preset first threshold as the semantic unit; a segmentation module for segmenting the semantic unit to form a target word segment; and a translation module for receiving the target word segment, translating, and outputting.

[0123] The voice recognition module transmits the recognized semantic elements to the semantic unit acquisition module to generate a qualified semantic unit. The semantic unit acquisition module then sends the semantic unit to the segmentation module for segmentation to facilitate translation, and finally the translation module outputs the translation result.

[0124] Refer to Figure 8 , in some embodiments, after generating the semantic unit, it is necessary to determine the segmentation position of the source text output by the voice recognition module.

[0125] In other embodiments, in order to further improve translation accuracy and avoid translation ambiguity, the device may also include an attention recognition module for obtaining the attention value of each semantic element in the semantic unit, such as a word, and determining the position where the word with the largest attention value appears.

[0126] refer to Fig. 9 In some embodiments, in order to further improve the translation accuracy, further processing is required for the literal units with translation risks. The system may also include a literal expansion module for expanding the literal units with translation risks to eliminate translation risks.

[0127] In some embodiments, when expanding the semantic unit, the semantic expansion module directly expands the current semantic unit only based on the discovery that the translation effect is not good enough until it forms a new semantic unit. Its essential purpose is to increase the number of semantic elements in the semantic unit. This process can be completed without the attention mechanism.

[0128] Fig.10 It is a schematic diagram showing the synchronous operation of the speech translation method according to an embodiment of the present application.

[0129] As can be seen from the figure, the first words 1, 2, and 3 received have been processed by all the previous modules and output to the translation module for translation. At the same time, the segmentation module is segmenting the newly added 4th word, the meaning unit acquisition module is judging the meaning unit of the 5th word accumulated in sequence, and the speech recognition module has started recognizing the 6th word. Therefore, the modules work synchronously, ensuring the speed and efficiency of translation, making it applicable in application scenarios such as simultaneous interpretation.

[0130] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising a processor and a memory:

[0131] Memory, used to store programs:

[0132] The processor is used to execute a program to perform the speech translation method of any one of the above embodiments.

[0133] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including:

[0134] The computer-readable storage medium stores a program, and the program is executed by a processor to complete the speech translation method of any of the above embodiments.

[0135] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual matching or direct fusion or communication connection shown or discussed can be through some interfaces, indirect fusion or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0138] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. The mobile terminal device may be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal device, a wearable device, a super mobile personal computer, a netbook, a personal digital assistant, a CPE, a UFI (wireless hotspot device), etc.; the embodiments of the present invention are not specifically limited.

[0139] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A speech translation method, comprising: Acquire a source speech signal, and identify meaningful elements in the source speech signal; sequentially accumulating at least one of the literal elements to form a judgment unit; Based on the BERT classification model, calculate the probability value of the judgment unit under the preset classification result condition; If the probability value is greater than a first threshold, the judgment unit is a meaningful unit; Segmenting the content units to form target word segments; Based on the Transformer translation model, the target word segment is translated and output; The segmenting of the literal units to form target segments includes: Obtaining the attention value of each literal element in the current literal unit, and determining the position where the literal element with the largest attention value appears, wherein the attention value is obtained through an attention mechanism model; If the content element with the largest attention value appears at a preset position of the content unit, segmenting the content unit to form a target word segment; If the literal element with the largest attention value appears in a non-preset position of the current literal unit, the current literal unit is expanded; Acquire the attention value of each of the literal elements in the expanded literal unit, and determine the position where the literal element with the largest attention value appears, until the literal element with the largest attention value appears at a preset position of the literal unit, and stop expanding; The final expanded meaning units are segmented to form target word segments.

2. The speech translation method according to claim 1, characterized in that: The calculating, based on the BERT classification model, the probability value of the judgment unit under the condition of determining the classification result includes: sequentially accumulating the literal elements that have not entered the judgment unit to form a preparatory unit, wherein the preparatory unit includes at least one literal element; Based on the BERT classification model, the judgment unit and the preparation unit are input, and the probability value of the judgment unit under the condition that the classification result is a literal unit is calculated.

3. The speech translation method according to claim 1, characterized in that: Calculating the accuracy of the target word segment translation based on the BLEU model, and if the accuracy is less than a second preset threshold, expanding the meaning unit, including: Based on the current literal unit, the literal elements are sequentially added until a new literal unit is formed.

4. The speech translation method according to any one of claims 1 to 3, characterized in that: The literal element is at least one of the following: a single word, a character, a word, a phrase, and an idiom.

5. A speech translation device, characterized in that: include: A speech recognition module, used to obtain a source speech signal and identify the literal elements in the source speech signal; A literal unit acquisition module, used to sequentially accumulate at least one literal element to form a judgment unit, calculate the probability value of the judgment unit under a preset classification result condition based on the BERT classification model, and select the judgment unit with a probability value greater than a first threshold as the literal unit; A segmentation module, used for segmenting the literal units to form target word segments; The translation module is used to receive the target word segment, translate it and output it; The segmenting of the literal units to form target segments includes: Obtaining the attention value of each literal element in the current literal unit, and determining the position where the literal element with the largest attention value appears, wherein the attention value is obtained through an attention mechanism model; If the content element with the largest attention value appears at a preset position of the content unit, segmenting the content unit to form a target word segment; If the literal element with the largest attention value appears in a non-preset position of the current literal unit, the current literal unit is expanded; Acquire the attention value of each of the literal elements in the expanded literal unit, and determine the position where the literal element with the largest attention value appears, until the literal element with the largest attention value appears at a preset position of the literal unit, and stop expanding; The final expanded meaning units are segmented to form target word segments.

6. An electronic device, comprising a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include a program for executing the speech translation method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The device stores program instructions executable by a processor, wherein the program instructions are used to execute the speech translation method according to any one of claims 1 to 4.

Images