Speech translation method, device, apparatus, and computer-readable storage medium

Abstract

This application discloses a speech translation method, apparatus, and storage medium. The method comprises: obtaining recognized text and at least one candidate translation result from speech data; performing accent recognition on the recognized text to obtain an accent recognition result; and obtaining a target translation result for the recognized text based on the accent recognition result and the at least one candidate translation result. Through this approach, the application can obtain highly accurate translation results.

Description

Technical Field

[0001] The present application relates to the field of translation, and in particular to a speech translation method, apparatus, device, and computer-readable storage medium. Background Art

[0002] With the rapid development of the information age, cross-language communication barriers have become a common problem faced by people around the world. Translation services are one of the main means of resolving language barriers. Traditional translation services are limited by manpower and costs, making them difficult to popularize. Emerging voice translation technology addresses these shortcomings in terms of cost and timeliness, providing a new way for ordinary people to communicate in daily life.

[0003] Speech translation refers to the process of automatically translating a speech signal in a source language into a target language. This process typically requires first translating the target language and then converting the translated result into a speech signal in the target language. The source language speech signal often emphasizes key words, manifesting as slow speech and accents. Existing translation solutions simply translate the recognized text after speech signal recognition, without considering the accent information present in the speech. Summary of the Invention

[0004] The main technical problem solved by this application is to provide a speech translation method, apparatus, device and computer-readable storage medium that can obtain translation results with high accuracy.

[0005] To solve the above technical problems, the first aspect of the present application provides a speech translation method, which includes: obtaining recognized text and at least one candidate translation result of speech data; performing accent recognition on the recognized text to obtain an accent recognition result; and obtaining a target translation result of the recognized text based on the accent recognition result and at least one candidate translation result.

[0006] To solve the above technical problems, the second aspect of the present application provides a speech translation device, which includes: an acquisition module for acquiring recognized text and at least one candidate translation result of speech data; an accent recognition module for performing accent recognition on the recognized text to obtain an accent recognition result; and a translation result determination module for obtaining a target translation result of the recognized text based on the accent recognition result and at least one candidate translation result.

[0007] To solve the above technical problems, the third aspect of the present application provides a speech translation device, which includes a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the speech translation method in the above first aspect.

[0008] In order to solve the above technical problems, the fourth aspect of the present application provides a computer-readable storage medium storing program instructions that can be executed by a processor, and the program instructions are used to implement the speech translation method in the above first aspect.

[0009] The beneficial effect of this application is that, unlike the existing technology, after obtaining the recognized text of speech data and at least one candidate translation result, this application determines the final target translation result of the recognized text based on the stress recognition result of the recognized text and the candidate translation results. This method considers the influence of stress factors on translation in existing translation solutions, thereby strengthening the stress information during the translation process and thus achieving a more accurate translation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a flowchart of the first embodiment of the speech translation method of the present application;

[0011] Figure 2 This is a flow chart of the second embodiment of the speech translation method of the present application;

[0012] Figure 3 This is a flowchart of the third embodiment of the speech translation method of the present application;

[0013] Figure 4 This is a flowchart of the fourth embodiment of the speech translation method of the present application;

[0014] Figure 5 This is a flow chart of an implementation of step S430 in the fourth implementation of the present application;

[0015] Figure 6 This is a flowchart of the fifth embodiment of the speech translation method of the present application;

[0016] Figure 7 This is a schematic diagram of the framework of an implementation method of the translation system provided by this application;

[0017] Figure 8 This is a flowchart of the sixth embodiment of the speech translation method of the present application;

[0018] Figure 9 This is a schematic diagram of the framework of an embodiment of the speech translation device provided by this application;

[0019] Figure 10 This is a schematic diagram of a framework of an embodiment of a speech translation device provided by this application;

[0020] Figure 11 It is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0021] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] It should be noted that in the embodiments of this application, there are descriptions involving "first", "second", etc., which are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" or "second" can explicitly or implicitly include at least one of such features. In addition, the participle mentioned in this application can be a single word or a word composed of multiple words.

[0023] See also Figure 1 , Figure 1 This is a flow chart of the first embodiment of the speech translation method of the present application. The method includes:

[0024] S110: Acquire the recognized text of the speech data and at least one candidate translation result.

[0025] In this embodiment, the recognized text can be obtained by performing speech recognition on the speech data. Specifically, the speech data can be divided into speech frames according to time slices, and the features of each speech frame are extracted to form a feature frame sequence to obtain feature data. The feature data is then fed into a pre-trained recognition model to obtain the recognized text.

[0026] After obtaining the recognized text, the recognized text is fed into a translation system, which generates one or more candidate translation results. The candidate translation results may be, for example, candidate translation texts or corresponding candidate translation speech data. The translation system may be a pre-trained translation model or other module capable of performing translation.

[0027] S120: Performing accent recognition on the recognized text to obtain an accent recognition result.

[0028] For example, the stress in the speech data can be identified to obtain the stressed words in the speech data, and then the stressed words in the speech data can be mapped to the recognized text to obtain the stress recognition results in the recognition file. The stress recognition results can specifically be, but are not limited to, the positions of the stressed words in the recognized text.

[0029] S130: Obtain a target translation result of the recognized text based on the accent recognition result and at least one candidate translation result.

[0030] In this embodiment, the target translation result of the recognized text can be determined with reference to the stress of the recognized text. The target translation result can be, for example, the target translation text or the corresponding target translation voice data. Since the influence of the stress factor on the translation is taken into consideration, the stress information in the translation process can be strengthened, thereby obtaining a translation result with higher accuracy.

[0031] Specifically, a target translation score for each candidate translation result may be determined based on the accent recognition result, and the target translation result may be determined using the target translation score of each candidate translation result. For example, a candidate translation result whose target translation score meets a first requirement may be selected from at least one candidate translation result as the target translation result, or at least two candidate translation results whose target translation scores meet a second requirement may be selected from at least one candidate translation result, and then the target translation result may be obtained by combining the at least two candidate translation results.

[0032] In a specific embodiment, the position of the accented words in the recognized text can be determined based on the accent recognition results, and a target translation score for each candidate translation result can be determined based on the position of the accented words. Specifically, for each candidate translation result, the translation accuracy of each accented word in the candidate translation result is obtained, and the target translation score for the candidate translation result is obtained using the translation accuracy of each accented word in the candidate translation result. For example, a first mean of the translation accuracy of each stressed word in the candidate translation result and a second mean of the translation accuracy of non-stressed words in the candidate translation result are obtained, and the first mean and the second mean are weighted averaged or weighted summed to serve as the target translation score of the candidate translation result, wherein the weight of the first mean is greater than the weight of the second mean; for another example, the translation accuracy of each segmentation in the candidate translation result is weighted summed or weighted averaged to serve as the target translation score of the candidate translation result, wherein the weight of the segmentation belonging to the stressed word is higher than the weight of the non-stressed word; for another example, the translation accuracy of each stressed word in the candidate translation result is directly counted using statistical methods such as variance and mean to obtain a statistical result, which is used as the target translation score of the candidate translation result.

[0033] In a specific application scenario, the incentive weight of each original word in the recognized text can be determined based on the accent recognition result, and the target translation score of each candidate translation result can be determined based on the incentive weight and the score factor of each candidate translation result.

[0034] Specifically, for example, the position of the accented words in the recognized text can be used to calculate the incentive weight of each original participle in the recognized text for each candidate translation result, and obtain the scoring factor of the candidate translation result, wherein the scoring factor of the candidate translation result includes at least one of the following: the translation accuracy of each translation participle in the candidate translation result, and the translation contribution of each original participle to each translation participle of the candidate translation result. By counting the incentive weights of each original participle and the scoring factors of the candidate translation result, the target translation score of the candidate translation result can be obtained. The statistical method can be, but is not limited to, product and sum calculation. The above method can improve the accuracy of the translation result by adding the incentive weights of each original participle in the recognized text to the original translation result and using the incentive weights to calculate the target translation score of the candidate translation result.

[0035] It is understood that the aforementioned translation accuracy and translation contribution can be, but are not limited to, provided by the translation system. In some specific application scenarios, the translation system includes an encoding module, a decoding module, and an attention calculation module. Translation accuracy can be provided directly by the translation system, while translation contribution is calculated by the translation system's attention calculation module. The attention calculation module determines the translation contribution of each word based on the output features obtained by the translation system through the decoding network and the decoded features of each word in the recognized text.

[0036] See also Figure 2 , Figure 2 : is a flow chart of the second embodiment of the speech translation method of the present application, which includes:

[0037] S210: Acquire the recognized text of the speech data and at least one candidate translation result.

[0038] Specifically, in one embodiment, the speech data is converted into recognized text by a speech recognition system, and then the recognized text is translated using a translation model or a translation system to obtain one or more translation results.

[0039] S220: Determine the incentive weight of each original segmentation based on the positional relationship between each original segmentation and the accented word.

[0040] In one embodiment, accented words in speech data can be identified using accent recognition technology, and the original word segmentation corresponding to the accented word in the recognized text can be marked. If the position of the original word segmentation overlaps with the accented word, it indicates that the original word segmentation is an accented word; if the position of the original word segmentation does not overlap with the accented word, the original word segmentation is not an accented word. The incentive weight of the original word segmentation that overlaps with the accented word is greater than the incentive weight of the original word segmentation that does not overlap with the accented word.

[0041] The method of determining the incentive weight of each original participle may include: determining the incentive weight of the original participle that overlaps with the position of the accent word as a first preset weight, and determining the incentive weight of the original participle that does not overlap with the position of the accent word as a second preset weight, wherein the first preset weight is greater than the second preset weight; or, for each original participle, determining the sub-weights of the original participle and each accent word, and statistically calculating the sub-weights corresponding to each accent word to obtain the incentive weight of the original participle, wherein the greater the distance between the original participle and the accent word, the smaller the corresponding sub-weight.

[0042] In a specific application scenario, the incentive weight of the original segmentation word that overlaps with the accented word position is determined to be the first preset weight, and the incentive weight of the original segmentation word that does not overlap with the accented word position is determined to be the second preset weight. For example, the first preset weight is set to 1 and the second preset weight is set to 0. It is understandable that in other application scenarios, the first preset weight and the second preset weight can also be set to other values.

[0043] In another specific application scenario, the incentive weight of the original word can be obtained by counting the sub-weights corresponding to each accented word. The following function can be used to calculate the incentive weight of the original word:

[0044]

[0045] Among them, θ i Represents the incentive weight for identifying the i-th original word in the text. It represents the sub-weight of each accented word in the recognized text corresponding to the i-th original segmentation, which can be calculated using the following formula:

[0046]

[0047] Among them, I k Indicates the position of the kth accent word in the text. For example, in one embodiment, the recognized text contains 5 original participles, of which the first accent word is the first original participle and the second accent word is the third original participle. Now calculate the incentive weight of the fourth original participle in the recognized text. The sub-weight of the fourth original participle corresponding to the first accent word is 1 / 16, and the sub-weight of the fourth original participle corresponding to the second accent word is 1 / 4. Adding the two sub-weights, the incentive weight of the fourth original participle can be obtained.

[0048] It is understandable that in other embodiments, other methods may be used to calculate the incentive weights of the original word segmentations, which are not limited here.

[0049] S230: Determine a target translation score for each candidate translation result based on the incentive weight and the score factor of each candidate translation result.

[0050] Specifically, by statistically analyzing the incentive weight of each word in the identified text and the score factor of each word in each candidate translation result, the target translation score of each candidate translation result can be obtained. The statistical method can be a product or sum method.

[0051] S240: Select the target translation result from at least one candidate translation result using the target translation score.

[0052] See also Figure 3 , Figure 3 : is a flowchart of the third embodiment of the speech translation method of the present application, the method comprising:

[0053] S310: Obtaining the recognized text of the speech data and at least one candidate translation result.

[0054] S320: Perform accent recognition on the speech data to obtain the probability that each speech frame in the speech data is an accent frame.

[0055] In the present embodiment, can but not be limited to obtain the probability that each voice frame belongs to accent frame by recognition model, in a specific embodiment, voice data can be divided into several frames, extract the feature of every frame to form feature frame sequence, obtain characteristic data.Characteristic data is input into recognition model, provide the probability that each frame belongs to accent frame by pattern recognition method.In other embodiments, also can adopt other mode to judge that voice frame belongs to the probability of accent frame, do not limit at this.

[0056] S330: For each original word segment, calculate the probability of each speech frame corresponding to the original word segment to obtain the stress score of the original word segment.

[0057] Specifically, in one embodiment, when judging whether the i-th original participle is an accent word in the recognition text, it is possible to determine that the i-th original participle appears in which speech frames according to the temporal correspondence, calculate the probability that all speech frames in which the i-th original participle occurs belong to the accent frame, and obtain the accent score of the i-th original participle. Wherein, the mode that all speech frames in which the i-th original participle occurs belong to the accent frame probability can be summed or averaged, or other statistical methods (such as Veterbi decoding algorithm) can be adopted, which is not limited here. The probability of each speech frame corresponding to each original participle in the recognition text is calculated respectively to obtain the accent score of each original participle.

[0058] S340: Based on the stress score, determine whether the original segmented word is a stressed word, and use the position of the original segmented word that is a stressed word as the position of the stressed word in the recognized text.

[0059] The stress score of the ith original participle is compared with the preset score. If the stress score of the ith original participle is greater than the preset score, the ith original participle is considered to be a stressed word.

[0060] S350: Determine the incentive weight of each original word in the recognized text based on the position of the accented word in the recognized text.

[0061] Specifically, the incentive weight of the original segmentation that overlaps with the position of the stress word in the recognition text is greater than the incentive weight of the original segmentation that does not overlap with the position of the stress word. For example, in one embodiment, the incentive weight of the original segmentation that overlaps with the position of the stress word can be set to 1, and the incentive weight of the original segmentation that does not overlap with the position of the stress word can be set to 0.

[0062] S360: Determine a target translation score for each candidate translation result based on the incentive weight and the score factor of each candidate translation result.

[0063] For the above steps S310 and S360 , please refer to steps S110 and S130 of the first embodiment respectively, which will not be described in detail here.

[0064] S370: Select the target translation result from at least one candidate translation result using the target translation score.

[0065] See also Figure 4 , Figure 4 : is a flowchart of the fourth embodiment of the speech translation method of the present application, the method comprising:

[0066] S410: Obtaining the recognized text of the speech data and at least one candidate translation result.

[0067] S420: Determine the incentive weight of each original word in the recognized text based on the accent recognition result of the recognized text.

[0068] For steps S410 and S420 , please refer to steps S110 and S120 of the first embodiment, and will not be described in detail here.

[0069] S430: Obtaining sub-scores corresponding to the word groups based on the target incentive weight, the target translation accuracy, and the target translation contribution.

[0070] In one embodiment, a segmentation group may be composed of the original segmentation words in the recognized text and the translated segmentation words in the candidate translation results. For each candidate translation result, different segmentation groups are formed by each original segmentation word and each translated segmentation word in the candidate translation result. For example, select the first translated segmentation word in the candidate translation result, and find the first original segmentation word in the recognized text corresponding to the first translated segmentation word, and form the first segmentation group with the first original segmentation word and the first translated segmentation word. It should be noted that since one original segmentation word in the recognized text may correspond to two translated segmentation words in the candidate translation result (for example, if the original segmentation word is "apple" and the translated segmentation word is "苹果", at this time the first translated segmentation word corresponding to the original segmentation word "apple" may be "苹", and the corresponding second translated segmentation word may be "果"), so the same original segmentation word may appear in multiple segmentation groups. Similarly, multiple original segmentation words may also appear in one segmentation group, and an original segmentation word described here is a single Chinese character.

[0071] The target incentive weight is the incentive weight of the original segmentation word in the segmentation group, that is, the incentive weight obtained in step S420; the target translation accuracy is the translation accuracy of the translated segmentation word in the segmentation group, and this translation accuracy can be directly obtained from the translation result of the recognized text through the translation system; the target translation contribution degree is the translation contribution degree of the original segmentation word in the segmentation group to the translated segmentation word in the segmentation group.

[0072] Statistically calculate the target incentive weight, the target translation accuracy, and the target translation contribution degree, and the sub-score of the segmentation group can be obtained. For example, in one embodiment, the first original segmentation word in the recognized text has been determined, as well as the corresponding first translated segmentation word of this original segmentation word in the candidate translation result, and the first original segmentation word and the first translated segmentation word are formed into the first segmentation group. According to the position of the stressed word in the recognized text, calculate the first target incentive weight of the first original segmentation word; and obtain the first target translation accuracy of the first translated segmentation word and the first target translation contribution degree of the first original segmentation word to the first translated segmentation word, and perform corresponding operations on the first target incentive weight, the first target translation accuracy, and the first target translation contribution degree, such as multiplication or addition, to obtain the first sub-score of the first segmentation group.

[0073] S440: Statistically calculate the sub-scores corresponding to each segmentation group to obtain the target translation score.

[0074] Use step S430 to calculate the sub-scores of all segmentation groups respectively, and sum up the sub-scores of all segmentation groups to obtain the target translation score.

[0075] S450: Use the target translation score to select the target translation result from at least one candidate translation result.

[0076] This embodiment forms word groups with the original word segments and the translated word segments, calculates the score of each word segment group, and then sums the scores of all word segment groups to obtain the target translation score of the candidate translation result. This can reduce errors in the calculation process and ensure the accuracy of the target translation score.

[0077] See also Figure 5 , Figure 5 This is a flow chart of an embodiment of step S430 in the fourth embodiment of the present application. Based on the target incentive weight, target translation accuracy and target translation contribution, the sub-score corresponding to the word group is obtained, including:

[0078] S431: Obtain the product of the target incentive weight and the target translation contribution to obtain a first value.

[0079] The target incentive weight refers to the incentive weight of an original segmentation in the segmentation group, and the target translation contribution refers to the translation contribution of an original segmentation in the recognition text to the translation segmentation when obtaining the translation segmentation in the translation result.

[0080] S432: Obtain the sum of the first value and the preset value to obtain a second value.

[0081] The preset value may be any integer, such as 1, 2, or 3. Preferably, the preset value may be set to 1.

[0082] S433: The product of the second value and the target translation accuracy is used as the sub-score corresponding to the word group.

[0083] In one embodiment, the preset value is set to 1, and the score of a word group can be obtained using the following formula:

[0084]

[0085] Among them, θ i is the target incentive weight, α ji The target translation contribution of the i-th original word when translating the j-th translation word, p(y j |X,y0,y1,…y j-1 ) is the target translation accuracy, N src To identify the number of original words in the text, N dst The number of translation words in the translation result.

[0086] For example, in a specific embodiment, the first word group contains two original word groups and one translated word group, and the first target incentive weight, the second target incentive weight, the first target translation contribution, the second target translation contribution and the target translation accuracy are obtained respectively. The product of the first target incentive weight and the first target translation contribution is added to a preset value to obtain a third value, the product of the second target incentive weight and the second target translation contribution is added to a preset value to obtain a fourth value, and the third value and the fourth value are added and multiplied by the target translation accuracy to obtain the sub-score of the first word group.

[0087] Please refer to Figure 6 and Figure 7 , Figure 6 This is a flowchart of the fifth embodiment of the speech translation method of the present application; Figure 7 This is a schematic diagram of a framework of an embodiment of the translation system provided by this application. The method further includes:

[0088] S610: Obtaining the recognized text of the speech data and at least one candidate translation result.

[0089] In one embodiment, the speech data may be converted into recognized text by a speech recognition system, and then the recognized text may be translated using a translation model to obtain one or more translation results.

[0090] S620: Determine the incentive weight of each original word in the recognized text based on the accent recognition result of the recognized text.

[0091] Among them, the stress recognition result includes the position of the stress word in the recognized text, and the incentive weight of each original word is determined according to the position of the stress word. In one embodiment, the closer the original word in the recognized text is to the position of the stress word, the greater the corresponding incentive weight.

[0092] S630: Obtain the translation accuracy corresponding to each candidate translation result from the translation result of the recognized text; and / or obtain the encoding feature of each original word segment and the decoding feature corresponding to each translated word segment.

[0093] Translation accuracy is a specific numerical value that can be, but is not limited to, obtained by the translation system. Encoding and decoding features are obtained during the translation of the recognized text. Using any original segmentation as the target original segmentation and any translated segmentation as the target translated segmentation, the encoding features of the target original segmentation and the decoding features of the previous translated segmentation of the target translated segmentation are used to determine the translation contribution of the target original segmentation to the target translated segmentation.

[0094] In a specific embodiment, the encoding features and decoding features can be obtained from an encoder-decoder based translation system, such as Figure 7As shown in Figure 1, the translation system mainly consists of three modules: encoding module (Encode), attention calculation module (Attention) and decoding module (Decode). After the recognition text enters the translation system, it first passes through the encoding module, which encodes the input recognition text to obtain the recognition text encoding information. The recognition text encoding information can be expressed as H = (h1,h2,h3,···,h Nsrc ), where H includes the encoding features of all original word segments in the identified text, h1 represents the encoding features of the first original word segment, h2 represents the encoding features of the second original word segment, and h Nsrc Indicates the encoding feature of the last original word of the recognized text, N src Indicates the length of the recognized text, that is, how many original words the recognized text contains.

[0095] The decoding module inputs the translation result of the previous translation word (such as y j-1 ), and obtain the corresponding output features (such as s j-1 ), using the encoding features of the target original segmentation and the decoding features of the previous translation segmentation of the target translation segmentation, the translation contribution of the target original segmentation to the target translation segmentation is obtained. For example, in one embodiment, the encoding features of the target original segmentation are h i , the decoding feature of the previous translation segmentation of the target translation segmentation is s j-1 , the following formula can be used to calculate the translation contribution of the i-th original participle to the j-th translated participle:

[0096]

[0097] Among them, exp(a(h i ,s j-1 )) refers to the exponential function with base e; a(h i ,s j-1 ) is the encoding feature h of the i-th original word i and the decoding feature S of the previous translation segmentation j-1 Function of N src Indicates the length of the recognized text; h k It is the encoding feature of the k original word segmentations in the identification text; It is the sum of the functions of each original word.

[0098] The above function a(h i ,s j-1 ) can be any of the following forms, where v and W are constant matrices, and d is h i length,

[0099] a(h i ,s j-1)=v T tanh(W h h i +W s s j-1 )

[0100] a(h i ,s j-1 )=h i T s j-1

[0101]

[0102] a(h i ,s j-1 )=h i T Ws j-1

[0103] It can be understood that the function a(h i ,s j-1 ) can also take other forms and are not limited here.

[0104] S640: Determine a target translation score for each candidate translation result based on the incentive weight and the score factor of each candidate translation result.

[0105] Among them, the scoring factors of the candidate translation results may include the translation accuracy corresponding to each candidate translation result and the translation contribution of each original segmentation to each translation segmentation in the candidate translation result. The target translation score of the candidate translation result can be obtained by statistically analyzing the incentive weight of the original segmentation, the translation accuracy corresponding to the candidate translation result, and the translation contribution of the original segmentation to each translation segmentation in the candidate translation result.

[0106] S650: Select a target translation result from at least one candidate translation result using the target translation score.

[0107] Please refer to Figure 7 and Figure 8 , Figure 8 : is a flowchart of the sixth embodiment of the speech translation method of the present application, the method further comprising:

[0108] S810: Divide the speech data into several speech frames, and extract features of each speech frame to obtain feature data of the speech data; recognize the feature data to obtain recognized text.

[0109] In this embodiment, the voice data is divided into voice frames according to time slices, and can be divided into frames every 10 milliseconds or every 40 milliseconds. The features of each voice frame are extracted to form a feature frame sequence to obtain feature data. The feature data can be an N*d matrix, where N is the number of voice frames and d is the feature dimension. The feature data is fed into a recognition model trained in advance to obtain recognized text, and the recognized text can further be processed in accordance with relevant procedures (such as number regularization, punctuation conformance processing, etc.). It can be understood that in other embodiments, other voice recognition methods can also be used, such as using a voice recognition system (ASR system) to recognize voice data to obtain recognized text. The specific voice recognition method is not limited here.

[0110] S820: Translate the recognized text using a translation system to obtain a translation result of the recognized text, wherein the translation result includes at least one candidate translation result.

[0111] Furthermore, the recognized text can be fed into a translation system, which will generate at least one candidate translation result and score the translation accuracy of each candidate translation result. Specifically, an Encoder-Decoder-based translation system can be used for translation, such as Figure 7 As shown in the figure, after the i-th original word in the recognition text passes through the encoding module, the recognition text encoding information h is obtained. i .

[0112] Input the translation result y of the previous word into the decoding module j-1 , obtain the corresponding output feature s through the decoding network j-1 The output features and the recognized text encoding information are input into the self-attention calculation module. The self-attention calculation module calculates the current attention information and combines it with the output features of the decoding module to obtain the current translation result. The translation result is obtained by translating all the original segmented words in the recognized text. During the translation process, the translation system can generate multiple candidate translation results, each of which is assigned a score.

[0113] The network structure included in the translation system can be implemented in a variety of ways, such as CNN (Convolutional Neural Networks), LSTM (Long Short-Term Memory), Self-Attention (self-attention mechanism), etc., which are not limited here.

[0114] S830: Determine the incentive weight of each original word in the recognized text based on the accent recognition result of the recognized text.

[0115] S840: Determine a target translation score for each candidate translation result based on the incentive weight and the score factor of each candidate translation result.

[0116] Steps S830 - S840 may refer to steps S120 - S130 of the first embodiment, and are not described again here.

[0117] S850: Selecting a target translation result corresponding to the recognized text from at least one candidate translation result based on the target translation score.

[0118] In one embodiment, after the target translation scores of all candidate translation results are obtained, the candidate translation result with the highest target translation score may be selected as the target translation result.

[0119] See also Figure 9 , Figure 9 This is a schematic diagram of the framework of one embodiment of the speech translation device provided in this application. The speech translation device 90 includes an acquisition module 91, a weight determination module 92, and a calculation module 93. Acquisition module 91 is used to acquire recognized text and at least one candidate translation result from speech data; accent recognition module 92 is used to perform accent recognition on the recognized text and obtain an accent recognition result; and translation result determination module 93 is used to obtain a target translation result for the recognized text based on the accent recognition result and at least one candidate translation result.

[0120] The method of obtaining a target translation result of the recognized text based on the accent recognition result and at least one candidate translation result includes: determining a target translation score for each candidate translation result based on the accent recognition result; and selecting a target translation result from the at least one candidate translation result using the target translation score.

[0121] The target translation score of each candidate translation result is determined based on the accent recognition result, including: determining an incentive weight for each original word in the recognized text based on the accent recognition result; and determining a target translation score for each candidate translation result based on the incentive weight and a score factor of each candidate translation result.

[0122] In some disclosed embodiments, the above-mentioned accent recognition results include the positions of accented words in the recognized text. Determining an incentive weight for each original word in the recognized text based on the accent recognition results specifically includes: determining an incentive weight for each original word based on the positional relationship between each original word and the accented word, wherein the incentive weight for an original word that overlaps with the accented word is greater than the incentive weight for an original word that does not overlap with the accented word.

[0123] Among them, based on the positional relationship between each original participle and the accent word, the incentive weight of each original participle is determined, including: determining the incentive weight of the original participle that overlaps with the accent word position as a first preset weight, and determining the incentive weight of the original participle that does not overlap with the accent word position as a second preset weight, wherein the first preset weight is greater than the second preset weight; or, for each original participle, determining the sub-weights of the original participle and each accent word, and statistically calculating the sub-weights corresponding to each accent word to obtain the incentive weight of the original participle, wherein the greater the distance between the original participle and the accent word, the smaller the corresponding sub-weight.

[0124] In some disclosed embodiments, the scoring factors of the candidate translation results include: the translation accuracy of each translated word in the candidate translation result, and / or the translation contribution of each original word to each translated word in the candidate translation result. Wherein, based on the incentive weight and the scoring factors of each candidate translation result, the target translation score of each candidate translation result is determined, specifically including: for each candidate translation result, each original word and each translated word in the candidate translation result are grouped into different word groups. For each word group, based on the target incentive weight, the target translation accuracy and the target translation contribution, the sub-score corresponding to the word group is obtained, wherein the target incentive weight is the incentive weight of the original word in the word group, the target translation accuracy is the translation accuracy of the translated word of the word group, and the target translation contribution is the translation contribution of the original word of the word group to the translated word of the word group, and the sub-score corresponding to each word group is counted to obtain the target translation score.

[0125] Among them, based on the target incentive weight, target translation accuracy and target translation contribution, the sub-score corresponding to the word group is obtained, including: obtaining the product of the target incentive weight and the target translation contribution, and taking the sum of the product and a preset value and the product of the target translation accuracy as the sub-score corresponding to the word group; counting the sub-scores corresponding to each word group to obtain the target translation score, including: summing the sub-scores corresponding to each word group to obtain the target translation score.

[0126] Among them, before determining the target translation score of each candidate translation result based on the incentive weight and the scoring factor of each candidate translation result, it includes: obtaining the translation accuracy corresponding to each candidate translation result from the translation result of the recognized text; and / or obtaining the encoding features of each original segmentation and the decoding features corresponding to each translated segmentation, wherein the encoding features and the decoding features are obtained in the process of translating the recognized text; using any original segmentation as the target original segmentation and any translated segmentation as the target translated segmentation, using the encoding features of the target original segmentation and the decoding features of the previous translated segmentation of the target translated segmentation, to obtain the translation contribution of the target original segmentation to the target translated segmentation.

[0127] Among them, stress recognition is performed on the recognized text to obtain the stress recognition result, including: performing stress recognition on the voice data to obtain the probability that each voice frame in the voice data is a stress frame; counting the probability of each voice frame corresponding to each original participle to obtain the stress score of each original participle; based on the stress score, determining whether the original participle is a stress word, and taking the position of the original participle belonging to the stress word as the position of the stress word in the recognized text.

[0128] Obtaining the recognized text and at least one candidate translation result of the speech data includes: dividing the speech data into a plurality of speech frames and extracting features of each speech frame to obtain feature data of the speech data; recognizing the feature data to obtain recognized text; and translating the recognized text using a translation system to obtain a translation result of the recognized text, wherein the translation result includes at least one candidate translation result.

[0129] See also Figure 10 , Figure 10 It is a schematic diagram of the framework of an embodiment of the speech translation device provided by this application.

[0130] The speech translation device 100 includes a memory 101 and a processor 102 coupled to each other. The memory 101 stores program instructions, and the processor 102 is configured to execute the program instructions to implement the steps described in any of the aforementioned touch detection method embodiments. Specifically, the speech translation device 100 may include, but is not limited to, a desktop computer, a laptop computer, a server, a mobile phone, a tablet computer, and the like.

[0131] Specifically, the processor 102 is used to control itself and the memory 101 to implement the steps in any of the above-mentioned touch detection method embodiments. The processor 102 can also be called a CPU (Central Processing Unit). The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 102 can be implemented by an integrated circuit chip.

[0132] See also Figure 11 , Figure 11It is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of the present application.

[0133] The computer-readable storage medium 110 stores program instructions 111 . When the program instructions 111 are executed by a processor, they are used to implement the steps in any of the above method embodiments.

[0134] The computer-readable storage medium 110 can specifically be a medium that can store computer programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or it can also be a server that stores the computer program. The server can send the stored computer program to other devices for execution, or it can also execute the stored computer program itself.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0136] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0138] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A speech translation method, characterized in that: The method comprises: Obtaining a recognized text of the speech data and at least one candidate translation result obtained by translating the recognized text; Performing accent recognition on the recognized text to obtain an accent recognition result; the accent recognition result includes the position of the accented words in the recognized text; Determining an incentive weight for each original segmentation based on a positional relationship between each original segmentation and the accented word in the recognized text; wherein the incentive weight for the original segmentation that overlaps with the accented word is greater than the incentive weight for the original segmentation that does not overlap with the accented word; Counting the incentive weight of each original word and the score factor of each translated word in each candidate translation result to determine a target translation score for each candidate translation result; A target translation result is selected from the at least one candidate translation result using the target translation score.

2. The method according to claim 1, characterized in that The determining of the incentive weight of each original segmentation word based on the positional relationship between each original segmentation word and the accented word in the recognized text includes: Determine the incentive weight of the original segmentation that overlaps with the accented word position as a first preset weight, and determine the incentive weight of the original segmentation that does not overlap with the accented word position as a second preset weight, wherein the first preset weight is greater than the second preset weight; Alternatively, for each of the original participles, the sub-weights of the original participle and each of the accented words are determined, and the sub-weights corresponding to each of the accented words are statistically analyzed to obtain the incentive weight of the original participle, wherein the greater the distance between the original participle and the accented word, the smaller the corresponding sub-weight.

3. The method according to claim 1, characterized in that The scoring factor of each translation segment in the candidate translation result includes at least one of the following: the translation accuracy of each translation segment in the candidate translation result, and the translation contribution of each original segment to each translation segment in the candidate translation result.

4. The method according to claim 3, characterized in that The calculating the incentive weight of each original word and the score factor of each translated word in each candidate translation result to determine the target translation score of each candidate translation result includes: For each candidate translation result, different segmentation groups are formed by each original segmentation and each translated segmentation in the candidate translation result; For each of the word segment groups, a sub-score corresponding to the word segment group is obtained based on a target incentive weight, a target translation accuracy, and a target translation contribution, wherein the target incentive weight is the incentive weight of the original word segment in the word segment group, the target translation accuracy is the translation accuracy of the translated word segment of the word segment group, and the target translation contribution is the translation contribution of the original word segment of the word segment group to the translated word segment of the word segment group; The sub-scores corresponding to the word groups are counted to obtain the target translation score.

5. The method according to claim 4, characterized in that The sub-score corresponding to the word group is obtained based on the target incentive weight, the target translation accuracy, and the target translation contribution, including: Obtaining the product of the target incentive weight and the target translation contribution, and multiplying the sum of the product and a preset value and the target translation accuracy as the sub-score corresponding to the word group; The statistical analysis of the sub-scores corresponding to the word groups to obtain the target translation score includes: The sub-scores corresponding to the word groups are summed to obtain the target translation score.

6. The method according to claim 4, characterized in that Before calculating the incentive weight of each original word and the score factor of each translated word in each candidate translation result to determine the target translation score of each candidate translation result, the method further includes: Obtaining the translation accuracy corresponding to each candidate translation result from the translation results of the recognized text; and / or, Obtain encoding features of each original segmentation and decoding features corresponding to each translated segmentation, wherein the encoding features and decoding features are obtained during the translation process of the recognized text; use any of the original segmentation as a target original segmentation and any of the translated segmentation as a target translated segmentation, and use the encoding features of the target original segmentation and the decoding features of the previous translated segmentation of the target translated segmentation to obtain the translation contribution of the target original segmentation to the target translated segmentation.

7. The method according to claim 1, characterized in that The performing accent recognition on the recognized text to obtain an accent recognition result includes: Performing accent recognition on the speech data to obtain a probability that each speech frame in the speech data is an accent frame; For each of the original participles, counting the probability of each of the speech frames corresponding to the original participle to obtain a stress score of the original participle; Based on the accent score, it is determined whether the original word is an accented word, and the position of the original word that is an accented word is used as the position of the accented word in the recognized text.

8. The method according to claim 1, characterized in that Acquiring recognized text of speech data and at least one candidate translation result obtained by translating the recognized text includes: Dividing the speech data into a plurality of speech frames and extracting features of each speech frame to obtain feature data of the speech data; recognizing the feature data to obtain the recognized text; The recognized text is translated using a translation system to obtain a translation result of the recognized text, wherein the translation result includes at least one candidate translation result.

9. A speech translation device, characterized in that: The device comprises: an acquisition module, configured to acquire a recognized text of the speech data and at least one candidate translation result obtained by translating the recognized text; An accent recognition module, configured to perform accent recognition on the recognized text to obtain an accent recognition result; the accent recognition result includes the position of the accented word in the recognized text; A translation result determination module is configured to determine an incentive weight for each original segmentation based on a positional relationship between each original segmentation and the accented word in the recognized text; calculate the incentive weight for each original segmentation and a score factor for each translated segmentation in each candidate translation result to determine a target translation score for each candidate translation result; and select a target translation result from the at least one candidate translation result using the target translation score; wherein the incentive weight for the original segmentation that overlaps with the accented word is greater than the incentive weight for the original segmentation that does not overlap with the accented word.

10. A speech translation device, characterized in that: comprising a memory and a processor coupled to each other, The memory stores program instructions; The processor is configured to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program instructions, and the program instructions can be executed to implement the method according to any one of claims 1 to 8.

Images