Simultaneous Interpretation Latency Detection Method and Related Devices, Electronic Devices, Storage Media

Abstract

The present application discloses a simultaneous translation delay detection method, related devices, electronic devices, and storage media. The simultaneous translation delay detection method includes: detecting the sentence simultaneous translation delay of each sentence pair during the simultaneous translation process; wherein, the sentence pair includes a first sentence in the source language and a second sentence in the target language, and the sentence simultaneous translation delay of the sentence pair includes the frame-level delay of the sentence pair; and statistically obtaining the passage simultaneous translation delay based on the sentence simultaneous translation delay. The above solution can automatically detect the simultaneous translation delay, which helps to significantly reduce the detection time and detection cost compared with manual detection.

Description

Technical Field

[0001] This application relates to the field of simultaneous interpretation technology, and particularly to a method for detecting simultaneous interpretation delay, related devices, electronic devices, and storage media. Background Art

[0002] In recent years, with the application of deep learning in the fields of speech and natural language processing, the accuracy of speech recognition has been continuously improving, and the translation effect of machine translation (such as in the directions of Chinese-English, English-Chinese, etc.) has also been continuously improved. Among them, machine translation has basically reached the level of human translation in large-scale corpora.

[0003] In the previous era of human simultaneous interpretation, the method of manual subjective scoring could be directly used to score from multiple dimensions such as the translation rate of core information, translation accuracy, pronunciation, tone, and delay. However, with the advent of machine simultaneous interpretation, it is impossible to use high-cost manual scoring for comparison every time when optimizing the effect. In view of this, how to automatically detect simultaneous interpretation delay has become an urgent problem to be solved. Summary of the Invention

[0004] The main technical problem to be solved by this application is to provide a method for detecting simultaneous interpretation delay, related devices, electronic devices, and storage media, which can automatically detect simultaneous interpretation delay and help significantly reduce the detection time and cost compared with manual detection.

[0005] To solve the above technical problem, in the first aspect of this application, a method for detecting simultaneous interpretation delay is provided, including: detecting the sentence simultaneous interpretation delay of each sentence pair during simultaneous interpretation; wherein, the sentence pair includes a first sentence in the source language and a second sentence in the target language, and the sentence simultaneous interpretation delay of the sentence pair includes the frame-level delay of the sentence pair; based on the sentence simultaneous interpretation delay, the passage simultaneous interpretation delay is statistically obtained.

[0006] To solve the above problem, in the second aspect of this application, a device for detecting simultaneous interpretation delay is provided, including a detection module and a statistical module. The detection module is used to detect the sentence simultaneous interpretation delay of each sentence pair during simultaneous interpretation. The sentence pair includes a first sentence in the source language and a second sentence in the target language, and the sentence simultaneous interpretation delay of the sentence pair includes the frame-level delay of the sentence pair; the statistical module is used to statistically obtain the passage simultaneous interpretation delay based on the sentence simultaneous interpretation delay.

[0007] To solve the above problem, in the third aspect of this application, an electronic device is provided, including a processor and a memory coupled to each other; the processor is used to execute the program instructions stored in the memory to implement the method for detecting simultaneous interpretation delay in the first aspect above.

[0008] To solve the above problem, in the fourth aspect of this application, a computer-readable storage medium is provided. The computer-readable storage medium stores program instructions that can be run by a processor, and the program instructions are used to implement the method for detecting simultaneous interpretation delay in the first aspect above.

[0009] In the above solution, the simultaneous translation delay of each sentence pair during the simultaneous translation process is detected, and the sentence pair includes the first sentence in the source language and the second sentence in the target language. The simultaneous translation delay of the sentence pair includes the frame-level delay of the sentence pair. Then, based on the simultaneous translation delay, the simultaneous translation delay of the passage is statistically obtained. On the one hand, during the simultaneous translation process, for the detection of the passage-level delay, by first detecting the simultaneous translation delay at the sentence level, the detection granularity of the simultaneous translation delay can be refined. Moreover, the simultaneous translation delay at the sentence level further includes the frame-level delay, so that the detection granularity can be further refined to the frame level, which is conducive to greatly improving the accuracy of the simultaneous translation delay detection. On the other hand, since there is no need for manual delay statistics, the machine can automatically detect the simultaneous translation delay during the simultaneous translation process, which helps to significantly reduce the detection time and detection cost compared with manual detection. Description of the Drawings

[0010] Figure 1 is a schematic flowchart of an embodiment of the synchronous delay detection method of the present application;

[0011] Figure 2 is a schematic diagram of the simultaneous translation delay in the synchronous delay detection method of the present application;

[0012] Figure 3 is a schematic flowchart of an embodiment for obtaining the frame-level delay;

[0013] Figure 4 is a schematic framework diagram of an embodiment of the simultaneous translation delay detection device of the present application;

[0014] Figure 5 is a schematic framework diagram of an embodiment of the electronic device of the present application;

[0015] Figure 6 is a schematic framework diagram of an embodiment of the computer-readable storage medium of the present application. Detailed Embodiments

[0016] The solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings of the specification.

[0017] In the following description, specific details such as specific system architectures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the present application.

[0018] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after. In addition, "multiple" in this article means two or more than two.

[0019] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of an embodiment of the simultaneous translation delay detection method of the present application. Specifically, the simultaneous translation delay detection method in this embodiment may include the following steps:

[0020] Step S11: Detect the simultaneous translation delay of each sentence pair during the simultaneous translation process.

[0021] In this embodiment, the sentence pair includes a first sentence in the source language and a second sentence in the target language. It should be noted that the first sentence and the second sentence in the sentence pair should have semantic association, such as the first sentence and the second sentence in the sentence pair can express the same meaning. Specifically, the source language and the target language can be set according to the specific simultaneous translation scenario. For example, the source language can be "English" and the target language can be "Chinese"; or, the source language can be "Chinese" and the target language can be "English"; or, the source language can be "English" and the target language can be "French", which is not limited here.

[0022] In an implementation scenario, the second sentence can be the translation sentence after the first sentence is translated from the source language to the target language. Exemplarily, taking the source language as Chinese and the target language as English as an example, the first sentence can be "Hello, he asks where the railway station is?", then the second sentence can be the translation sentence after this first sentence is translated into English, "Hello, he asks the railway station, where is it?". Other situations can be deduced by analogy, and no further examples will be given here. Other situations can be deduced by analogy, and no further examples will be given here.

[0023] In an implementation scenario, both the first sentence and the second sentence can be voices. Similarly, taking the source language as Chinese and the target language as English as an example, at this time, the first sentence can be the Chinese voice of "Hello, he asks where the railway station is?", and the second sentence can be the English voice of "Hello, he asks the railway station, where is it?". Other situations can be deduced by analogy, and no further examples will be given here.

[0024] In another implementation scenario, both the first sentence and the second sentence can also be texts. Similarly, taking the source language as Chinese and the target language as English as an example, at this time, the first sentence can be the Chinese text of "Hello, he asks where the railway station is?", and the second sentence can be the English text of "Hello, he asks the railway station, where is it?". Other situations can be deduced by analogy, and no further examples will be given here.

[0025] In yet another implementation scenario, the first statement and the second statement can also have different data forms. For example, the first statement can be speech, and the second statement can be text; or, the first statement can be text, and the second statement can be speech. Similarly, reference can be made to the foregoing embodiments, and no further examples will be given here.

[0026] In one implementation scenario, the simultaneous interpretation delay of statements can include frame-level delay, which reflects the overall delay of each statement frame during the simultaneous interpretation process. Specifically, the frame-level delay can be obtained based on the delay times of each statement frame pair in the statement pair. The statement frame pair includes the first statement frame in the first statement and the second statement frame in the second statement. It should be noted that a frame can be regarded as the smallest measurement unit for simultaneous interpretation delay detection. Demonstratively, statement frames can be divided at a certain time interval. For example, a 1-second simultaneous interpretation process can be divided into 100 frame simultaneous interpretations, that is, the period of one frame simultaneous interpretation is 10 milliseconds.

[0027] In a specific implementation scenario, as described above, both the first statement and the second statement in the statement pair can be speech. In this case, taking Chinese as the source language and English as the target language as an example, the first statement frame in the statement frame pair can be a 10-millisecond segment of Chinese speech in "Hello, he asks where the railway station is?", and the second statement frame in the statement frame pair can be the corresponding 10-millisecond segment of English speech in "Hello, he asks the railway station, where is it?". Other cases can be inferred by analogy, and no further examples will be given here.

[0028] In another specific implementation scenario, as described above, both the first statement and the second statement in the statement pair can also be text. In this case, taking Chinese as the source language and English as the target language as an example, the first statement frame in the statement frame pair can be a 10-millisecond segment of Chinese text in "Hello, he asks where the railway station is?", and the second statement frame in the statement frame pair can be the corresponding 10-millisecond segment of English text in "Hello, he asks the railway station, where is it?". Other cases can be inferred by analogy, and no further examples will be given here.

[0029] In yet another specific implementation scenario, as described above, when the first statement in the statement pair is speech and the second statement is text, or when the first statement in the statement pair is text and the second statement is speech, in the above two cases, the first statement frame and the second statement frame in the statement frame pair can refer to the relevant descriptions of the first statement frame and the second statement frame in the statement frame pair when the first statement and the second statement have the same data form, and no further examples will be given here.

[0030] In an implementation scenario, the first statement frame in the statement frame pair is the statement frame numbered with the first ordinal number in the first statement, the second statement frame is the statement frame numbered with the second ordinal number in the second statement, and the first ordinal number and the second ordinal number satisfy a frame mapping relationship.

[0031] In a specific implementation scenario, the total number of the first statement frames in the first statement can be denoted as the first frame number, and the total number of the second statement frames in the second statement can be denoted as the second frame number. The frame mapping relationship can specifically represent a linear function, and the slope of the linear function can be the ratio of the second frame number to the first frame number. In addition, the intercept of the linear function can be set to 0, and at this time, the ratio of the second ordinal number to the first ordinal number is equal to the ratio of the second frame number to the first frame number. Of course, the intercept of the linear function can also be set to other values, and the frame mapping relationship can even be represented as other functions, which are not specifically limited in this embodiment. It should be noted that the frame mapping relationship should preferably be set as a one-to-one mapping relationship, that is, according to the first ordinal number and the frame mapping relationship, the second ordinal number can be uniquely determined, or according to the second ordinal number and the frame mapping relationship, the first ordinal number can be uniquely determined. In particular, when the first ordinal number (or the second ordinal number) determined according to the frame mapping relationship is a decimal, a ceiling operation can be performed. For example, if the frame mapping relationship is a linear function with a slope of 2 and an intercept of 0, when the second ordinal number is 5, the first ordinal number can be deduced to be 2.5 at this time, which is obviously unreasonable. In such cases, it should be understood that the statement frames of the second statement with the second ordinal number of 5 and the second ordinal number of 6 together form a statement frame pair with the statement frame of the first statement with the first ordinal number of 3. Other cases can be deduced by analogy and will not be exemplified one by one here.

[0032] In a specific implementation scenario, for the second statement frame numbered with the second ordinal number in the second statement, the first ordinal number can be determined based on the second ordinal number and the obtained frame mapping relationship, and then the first statement frame numbered with the first ordinal number and the second statement frame numbered with the second ordinal number can be formed into a statement frame pair. On this basis, the time consumed for the input of the first statement frame can be further obtained as the first time and the time consumed for the output of the second statement frame can be obtained as the second time, and the difference between the second time and the first time can be obtained, and this difference can be used as the delay time of the statement frame pair. Exemplarily, the frame mapping relationship between the second ordinal number and the first ordinal number is that the ratio of the second ordinal number to the first ordinal number is 1 and the intercept is 0. If the second ordinal number is 40, then the first ordinal number is 40; if the time consumed for the output of the second statement frame numbered 40 in the second statement is 110 milliseconds and the time consumed for the input of the first statement frame numbered 40 in the first statement is 100 milliseconds, then the delay time of this statement frame pair is 10 milliseconds. Other cases can be deduced by analogy and will not be exemplified one by one here.

[0033] In an implementation scenario, the simultaneous interpretation delay of a statement may also include an accumulation delay, which can be obtained based on the difference between the second frame count of the second statement and the first frame count of the first statement. As described above, the second frame count refers to the total number of second statement frames in the second statement, and the first frame count refers to the total number of first statement frames in the first statement.

[0034] In a specific implementation scenario, as described above, if both the first statement and the second statement in the statement pair are in speech, then the product of the difference between the above-mentioned first frame count and the second frame count and the period of the statement frame (e.g., the aforementioned 10 milliseconds) can be used as the accumulation delay. Please refer to Figure 2 , Figure 2 FIG. is a schematic diagram of the simultaneous interpretation delay of a statement in the method for detecting the simultaneous interpretation delay of this application. This schematic diagram takes the first statement and the second statement both being in speech, with the source language being Chinese and the target language being English as an example. As Figure 2 shown, as indicated by the arrows, they are respectively the spectrogram of the first statement, the recognition result of the first statement, the translation result of the first statement, the spectrogram of the second statement, the time length of the first statement, and the time length of the second statement. The cumulative delay is the difference between the time length of the second statement and the time length of the first statement.

[0035] In another specific implementation scenario, as described above, if both the first statement and the second statement in the statement pair are in text, also taking the source language being Chinese and the target language being English as an example, at this time, the difference between the product of the total number of English words in the second statement text and the period of each English word and the product of the total number of Chinese words in the first statement text and the period of the Chinese word can be used as the accumulation delay. It should be noted that depending on the size of the frame data volume and the language difference, each word in each language may be composed of several frames with different numbers, so there may be a situation where the total number of words in the second statement is less than the total number of words in the first statement.

[0036] In yet another specific implementation scenario, as described above, when the first statement in the statement pair can be in speech and the second statement can be in text, or when the first statement in the statement pair can be in text and the second statement can be in speech, in this case, for the specific calculation process of the accumulation delay, reference can be made to the relevant description of the accumulation delay when the first statement and the second statement in the statement pair have the same data form, and no examples will be given here for further elaboration. Exemplarily, when the first statement is in speech and the second statement is in text, also taking the source language being Chinese and the target language being English as an example, at this time, the cumulative delay is the difference between the time length of the output of the second statement English text and the time length of the input of the first statement Chinese text.

[0037] In an implementation scenario, after obtaining the accumulation delay, it can be determined whether the accumulation delay is greater than a first threshold, and based on the determination result, the speed of the simultaneous interpretation process of the statement can be adjusted.

[0038] In a specific implementation scenario, the first threshold can be set to 0. When it is determined that the cumulative delay is greater than 0, it can be determined that the second frame number of the second statement is too large. Then, the second statement in the next statement pair during the simultaneous interpretation process can be further adjusted. The specific adjustment methods can include, but are not limited to, statement reorganization, statement compression, etc., which are not limited here. Of course, the first threshold can also be set to other values according to actual needs, such as 1 millisecond, 5 milliseconds, etc., which are not limited here. Exemplarily, when both the first statement and the second statement are in speech, with the source language being Chinese and the target language being English, taking the statement reorganization method as an example, it can be adjusted from "Hello, he asks the railway station, where is it?" to "Hello, he asks where the railway station is?"; taking the statement compression method as an example, the voice of "Hello, he asks the railway station, where is it?" needs to be sent quickly.

[0039] In another specific implementation scenario, as mentioned above, the first threshold can be set to 0. When it is determined that the cumulative delay is less than 0, it can be determined that the second frame number of the second statement is too small. Then, the second statement in the next statement pair during the simultaneous interpretation process can be adjusted. The specific adjustment methods can include, but are not limited to, adding filler words, adding pauses, etc., which are not specifically limited in this embodiment. Exemplarily, when both the first statement and the second statement are in speech, with the source language being Chinese and the target language being English, taking adding a pause as an example, it can be adjusted from "Hello, he asks the railway station, where is it?" to "Hello~, he asks the railway station, where is it?"; taking adding filler words as an example, it can be adjusted from "Hello, he asks the railway station, where is it?" to "Hey, hello, he asks the railway station, where is it?"

[0040] In yet another specific implementation scenario, as mentioned above, the first threshold can be set to 0. When it is determined that the cumulative delay is equal to 0, it can be determined that the current speed of the simultaneous interpretation process of the statement is reasonable and no adjustment is required.

[0041] In yet another specific implementation scenario, different from the aforementioned method of setting the first threshold, a closed numerical range can also be preset. When the cumulative delay is within this numerical range, there is no need to adjust the second sentence in the next sentence pair during the simultaneous translation process, that is, the second sentence can be directly output; when the cumulative delay is greater than the right endpoint of the closed numerical interval, the second frame number of the second sentence needs to be reduced; when the cumulative delay is less than the left endpoint of the closed numerical interval, the second frame number of the second sentence needs to be increased. The adjustment method can refer to the above embodiments. The closed numerical interval can be set to [-10, 10], [-5, 5], [-5, 10], etc., and this embodiment does not make specific limitations here.

[0042] In one implementation scenario, the simultaneous translation delay of a sentence can also include a sentence start delay. Please continue to refer to Figure 2 , where the sentence start delay refers to the time difference between the output of the first frame of the second sentence and the input of the first frame of the first sentence.

[0043] In a specific implementation scenario, when both the first sentence and the second sentence are texts, taking Chinese as the source language and English as the target language as an example, the sentence start delay is then the time difference between the output of the first English text of the second sentence and the input of the first Chinese text of the first sentence.

[0044] In another specific implementation scenario, when the first sentence is speech and the second sentence is text; or when both the first sentence and the second sentence are speech. Similarly, the calculation method of the sentence start delay can refer to the foregoing embodiments, and no examples will be given here for elaboration. Exemplarily, when the first sentence is speech and the second sentence is text, taking Chinese as the source language and English as the target language, the sentence start delay is then the time difference between the output of the first English text of the second sentence and the input of the first frame of the first sentence.

[0045] In one implementation scenario, the simultaneous translation delay of a sentence can also include a sentence end delay. Please continue to refer to Figure 2 , where the sentence end delay refers to the time difference between the output of the last frame of the second sentence and the input of the last frame of the first sentence.

[0046] In a specific implementation scenario, when both the first sentence and the second sentence are texts, taking Chinese as the source language and English as the target language as an example, the sentence end delay is then the time difference between the output of the last English text of the second sentence and the input of the last Chinese text of the first sentence.

[0047] In another specific implementation scenario, when the first statement is in speech and the second statement is in text; or when both the first statement and the second statement are in speech. Similarly, the calculation method of the end-of-sentence delay can refer to the foregoing embodiments, and no further examples will be given here. Exemplarily, when the first statement is in speech and the second statement is in text, with the source language being Chinese and the target language being English, the end-of-sentence delay is the time difference between the output of the last English text of the second statement and the input of the last frame of the first statement.

[0048] It should be noted that the above simultaneous interpretation delays for statements can be implemented separately. For example, only the frame-level delay of each statement pair in the simultaneous interpretation process can be detected, or only the cumulative delay of each statement pair in the simultaneous interpretation process can be detected, or only the start-of-sentence delay of each statement pair in the simultaneous interpretation process can be detected, or only the end-of-sentence delay of each statement pair in the simultaneous interpretation process can be detected. In one implementation scenario, the above simultaneous interpretation delays for statements can also be combined. For example, the frame-level delay, cumulative delay, and start-of-sentence delay of each statement pair in the simultaneous interpretation process can be detected simultaneously, or the frame-level delay, cumulative delay, start-of-sentence delay, and end-of-sentence delay of each statement pair in the simultaneous interpretation process can be detected simultaneously, and so on. This embodiment does not make specific restrictions here.

[0049] As mentioned above, the simultaneous interpretation process of a passage can be decomposed into the simultaneous interpretation process of each statement pair by using a semantic unit segmentation module. In a real scenario, the simultaneous interpretation process is generally divided into two types, including the simultaneous interpretation process of outputting recognition results and the simultaneous interpretation process of not outputting recognition results, which can refer to Figure 2 , and the recognition result is the recognition result of the first statement. The difference between the two is whether the text information of the source language identified can be output. The simultaneous interpretation process of outputting recognition results has the function of outputting recognition, and the simultaneous interpretation process of not outputting recognition results does not have the function of recognizing and outputting results. Therefore, when segmenting the simultaneous interpretation of a passage, it is necessary to segment the two different simultaneous interpretation processes separately.

[0050] In one implementation scenario, taking the first statement and the second statement both being in speech as an example, in the simultaneous interpretation process of outputting recognition results, the end-of-sentence delimiter of the recognition result can be directly used for segmentation, and subsequent translation and synthesis are based on this segmentation identifier, so that the sentence-level alignment information from the first statement speech to the second statement speech can be achieved; in the simultaneous interpretation process of not outputting recognition results, the punctuation information of the translation of the second statement can be used for segmentation, and it is natural to obtain the word-level alignment information from the text to the synthesized speech during the synthesis process, so the sentence-level alignment information from the first statement speech to the second statement speech can also be obtained.

[0051] Step S12: Based on the simultaneous interpretation delay of the statements, the simultaneous interpretation delay of the passage is statistically obtained.

[0052] In an implementation scenario, it is possible to count the simultaneous interpretation latency of all the sentence pairs segmented by consecutive interpretation of a passage, and calculate the arithmetic mean of the simultaneous interpretation latencies of all the sentence pairs, which is the consecutive interpretation latency of the passage.

[0053] In another implementation scenario, it is possible to count the simultaneous interpretation latencies of the first sentence pair and the last sentence pair segmented by consecutive interpretation of a passage, and the average of the simultaneous interpretation latencies of these two sentence pairs is the consecutive interpretation latency of the passage.

[0054] In yet another implementation scenario, it is possible to count the simultaneous interpretation latencies of all the sentence pairs segmented by consecutive interpretation of a passage, and calculate the passage latency according to a preset proportional relationship. Specifically, a consecutive interpretation of a passage can be segmented into six sentence pairs for simultaneous interpretation. Assuming that the proportion of the first sentence pair and the last sentence pair is 30%, and the proportion of the remaining four sentence pairs is 10%, then the cumulative sum of 30% of the simultaneous interpretation latency of the first sentence pair, 30% of the simultaneous interpretation latency of the last sentence pair, and 10% of the simultaneous interpretation latency of the remaining sentence pairs is the consecutive interpretation latency of the passage.

[0055] In an implementation scenario, it is also possible to evaluate the latency of the simultaneous interpretation process based on the consecutive interpretation latency data. Specifically, certain score coefficients can be assigned to various passage latency data, calculate the score of the consecutive interpretation latency, and compare it with a preset score threshold to further evaluate the consecutive interpretation latency. For example, the values of frame-level latency, cumulative latency, sentence-start latency, and sentence-end latency in the consecutive interpretation latency are all 5; the score coefficient of frame-level latency is 5, and the score coefficients of cumulative latency, sentence-start latency, and sentence-end latency are all 3; the preset score threshold is 80; the score of the consecutive interpretation latency is 5×5 + 5×3×3 = 70, which is less than 80, so the consecutive interpretation latency meets the requirements.

[0056] In an implementation scenario, it is also possible to determine whether to continue the simultaneous interpretation process based on the simultaneous interpretation latency data of the sentence pair. A preset condition for frame-level latency can be set. If the frame-level latency of the current sentence pair meets the preset condition, the simultaneous interpretation can continue; otherwise, this simultaneous interpretation is terminated. Demonstratively, the preset condition for frame-level latency is set to be less than 10 frames. The frame-level latency of each sentence pair is judged to determine whether to continue the simultaneous interpretation process. If the frame-level latency of the current sentence pair is greater than or equal to 10 frames, this simultaneous interpretation process is terminated; if the frame-level latency of the current sentence pair is less than 10 frames, this simultaneous interpretation process can continue.

[0057] In the above solution, the simultaneous translation delay of each sentence pair during the simultaneous translation process is detected, and the sentence pair includes the first sentence in the source language and the second sentence in the target language. The simultaneous translation delay of the sentence pair includes the frame-level delay of the sentence pair. Then, based on the simultaneous translation delay, the simultaneous translation delay of the passage is statistically obtained. On the one hand, during the simultaneous translation process, for the detection of the passage-level delay, by first detecting the simultaneous translation delay at the sentence level, the detection granularity of the simultaneous translation delay can be refined, and the simultaneous translation delay at the sentence level further includes the frame-level delay, so that the detection granularity can be further refined to the frame level, which is conducive to greatly improving the accuracy of the simultaneous translation delay detection. On the other hand, since there is no need for manual delay statistics, the machine can automatically detect the simultaneous translation delay during the simultaneous translation process, which helps to significantly reduce the detection time and detection cost compared with manual detection.

[0058] Please refer to Figure 3 , Figure 3 which is a schematic flowchart of an embodiment for obtaining the frame-level delay. For the specific meaning of the frame-level delay, reference can be made to the relevant descriptions in the foregoing disclosed embodiments, which will not be elaborated herein. The embodiments of the present disclosure may specifically include the following steps:

[0059] Step S31: Obtain the sentence judgment result of the sentence pair.

[0060] In the embodiments of the present disclosure, the sentence judgment result includes whether the sentence pair is the first pair of simultaneous translation sentences in the simultaneous translation process.

[0061] Step S32: Based on the sentence judgment result of the sentence pair and the delay time of each sentence frame pair in the sentence pair, obtain the frame-level delay of the sentence pair.

[0062] Specifically, the influence factor of the sentence pair can be obtained based on the sentence judgment result. The influence factor represents the degree of influence of the reference sentence pair of the current sentence pair on the simultaneous translation delay of the current sentence pair. The reference sentence pair can be set as a pair of sentence pairs that have completed the simultaneous translation process immediately before the current sentence pair. The influence factor at least includes the influence duration of the reference sentence pair on the simultaneous translation delay of the current sentence pair, and may also include the influence rate, influence accuracy, etc. of the reference sentence pair on the simultaneous translation delay of the current sentence pair.

[0063] In an implementation scenario, when the sentence judgment result is that the sentence pair is the first pair of simultaneous translation sentences in the simultaneous translation process, the influence factor is set to a preset value. In an implementation scenario, the preset value can be set to 0.

[0064] In another implementation scenario, when the sentence judgment result is that the sentence pair is not the first pair of simultaneous translation sentences in the simultaneous translation process, the influence factor can be obtained based on the end delay of the previous sentence pair and the start delay of the current sentence pair. The obtaining methods of the end delay and the start delay have been introduced in the foregoing embodiments and will not be elaborated herein.

[0065] In an implementation scenario, calculate the difference between the head delay of the current statement pair and the tail delay of the previous statement, compare it with 0, and take the larger value as the influence factor.

[0066] The overall delay is the sum of the delay times of all statement frame pairs in the simultaneous translation process. Calculate the ratio of the overall delay to the total number of frames in the current statement pair, and then sum it with the influence factor to obtain the frame-level delay of the current statement pair. The method for obtaining the delay time of the statement frame pair has been introduced in the foregoing embodiments and will not be elaborated here.

[0067] In an implementation scenario, when the statement judgment result is that the statement pair is the first pair of simultaneous translation statements in the simultaneous translation process, the frame-level delay can be expressed as:

[0068]

[0069]

[0070] In the above formulas (1) and (2), DA is the frame-level delay, N is the number of frames per second of the second statement, |Y| is the time length of the second statement, |X| is the time length of the first statement, and d i represents the number of frames of the first statement input when the i-th frame of the second statement is output. It should be noted that γ is only one implementation manner of the frame mapping relationship and does not limit the implementation manner of the frame mapping relationship. For the specific meaning of the frame mapping relationship, reference can be made to the relevant descriptions in the foregoing disclosed embodiments.

[0071] In an implementation scenario, when the statement judgment result is that the statement pair is not the first pair of simultaneous translation statements in the simultaneous translation process, the frame-level delay can also be expressed as:

[0072]

[0073]

[0074] λ = max(0, β - α) ……(5)

[0075] In the above formulas (3), (4) and (5), DA is the frame-level delay, N is the number of frames per second of the second statement, |Y| is the time length of the second statement, |X| is the time length of the first statement, and d i represents the number of frames of the first statement input when the i-th frame of the second statement is output, α is the tail delay of the previous statement pair, and β is the head delay of the current statement pair. Among them, γ is only one implementation manner of the frame mapping relationship, and λ is only one implementation manner of the influence factor, and does not limit the implementation manners of the frame mapping relationship and the influence factor. For the specific meanings of the frame mapping relationship and the influence factor, reference can be made to the relevant descriptions in the foregoing disclosed embodiments.

[0076] For the above solution, the statement judgment result of the statement pair is obtained, and based on the statement judgment result of the statement pair and the latency time of each statement frame pair in the statement pair, the frame-level latency of the statement pair is obtained. On the one hand, it is judged whether the statement pair is the first statement pair, fully considering the influence of the simultaneous translation process of the previous statement pair on the simultaneous translation process of the current statement pair. On the other hand, the frame-level latency of the simultaneous translation process is detected, which can greatly improve the accuracy of simultaneous translation latency detection.

[0077] Please refer to Figure 4 , Figure 4 FIG. is a schematic framework diagram of an embodiment of the simultaneous translation latency detection device 40 of the present application. Specifically, the simultaneous translation latency detection device 40 includes a detection module 41 and a statistics module 42. The detection module 41 is used to detect the statement simultaneous translation latency of each statement pair during the simultaneous translation process. Among them, the statement pair includes a first statement in the source language and a second statement in the target language, and the statement simultaneous translation latency of the statement pair includes the frame-level latency of the statement pair; the statistics module 42 is used to statistically obtain the passage simultaneous translation latency based on the statement simultaneous translation latency.

[0078] For the above solution, the statement simultaneous translation latency of each statement pair during the simultaneous translation process is detected, and the statement pair includes a first statement in the source language and a second statement in the target language. The statement simultaneous translation latency of the statement pair includes the frame-level latency of the statement pair. Then, based on the statement simultaneous translation latency, the passage simultaneous translation latency is statistically obtained. On the one hand, during the simultaneous translation process, for the detection of passage-level latency, by first detecting the statement-level simultaneous translation latency, the detection granularity of the simultaneous translation latency can be refined, and the statement-level simultaneous translation latency further includes the frame-level latency, so that the detection granularity can be further refined to the frame level, which can greatly improve the accuracy of simultaneous translation latency detection. On the other hand, since there is no need for manual latency statistics, the machine can automatically detect the simultaneous translation latency during the simultaneous translation process, which helps to significantly reduce the detection time and detection cost compared with manual detection.

[0079] In some publicly disclosed embodiments, the frame-level latency is obtained based on the latency time of each statement frame pair in the statement pair. The statement frame pair includes a first statement frame in the first statement and a second statement frame in the second statement.

[0080] Therefore, the above solution discloses a method for obtaining the frame-level latency and the composition of the statement frame pair, and the frame-level latency can be obtained from the latency situation of the statement frame pair, which greatly improves the accuracy of simultaneous translation latency monitoring.

[0081] In some publicly disclosed embodiments, the detection module 41 includes a statement judgment sub-module for obtaining the statement judgment result of the statement pair. Among them, the statement judgment result includes whether the statement pair is the first pair of simultaneous translation statements in the simultaneous translation process; the detection module 41 includes a latency measurement sub-module for obtaining the frame-level latency of the statement pair based on the statement judgment result of the statement pair and the latency time of each statement frame pair in the statement pair.

[0082] Therefore, the above solution further introduces the concept of frame-level delay on the basis of detecting simultaneous interpretation of sentences. By detecting the frame-level delay in the process of simultaneous interpretation, it is beneficial to greatly improve the accuracy of detecting simultaneous interpretation delay.

[0083] In some disclosed embodiments, the delay measurement sub-module includes an influence factor acquisition unit for obtaining the influence factor of a sentence pair based on the sentence judgment result, where the influence factor represents the degree of influence of the reference sentence pair of the sentence pair on the simultaneous interpretation delay caused to the sentence pair.

[0084] Therefore, the above solution introduces the concept of influence factor based on the sentence judgment result, reflecting the influence of the reference sentence pair on the simultaneous interpretation of the sentence pair, making the factors considered in the simultaneous interpretation delay detection method more comprehensive and the detection more accurate.

[0085] In some disclosed embodiments, the influence factor acquisition unit is specifically configured to set the influence factor to a preset value in response to the sentence judgment result including that the sentence pair is the first pair of simultaneously interpreted sentences; and / or, the influence factor acquisition unit is specifically configured to, in response to the sentence judgment result including that the sentence pair is not the first pair of simultaneously interpreted sentences, obtain the influence factor based on the end delay of the reference sentence pair and the start delay of the sentence pair.

[0086] Therefore, the above solution reveals different calculation methods of the influence factor under different sentence judgment results, which helps to accurately calculate the frame-level delay.

[0087] In some disclosed embodiments, the statistics module 42 includes a delay time statistics sub-module for statistically summing up the delay times of sentence frame pairs to obtain the overall delay in the process of simultaneous interpretation; the delay measurement sub-module is used to obtain the frame-level delay based on the ratio of the overall delay to the number of sentence frame pairs and the influence factor.

[0088] Therefore, the above solution specifically introduces a way to obtain the frame-level delay. On the one hand, it determines whether the sentence pair is the first sentence pair, fully considering the influence of the simultaneous interpretation process of the previous sentence pair on the simultaneous interpretation process of the current sentence pair. On the other hand, it detects the frame-level delay in the process of simultaneous interpretation, which is beneficial to greatly improve the accuracy of detecting simultaneous interpretation delay.

[0089] In some disclosed embodiments, the delay measurement sub-module includes a sentence frame pairing unit for determining a second ordinal number based on the frame mapping relationship and the first ordinal number of the first sentence frame in the first sentence; and forming the sentence frame pair by the first sentence frame with the first ordinal number and the second sentence frame with the second ordinal number.

[0090] Therefore, the above solution points out the composition of the sentence frame pair. The sentence frame pair is the smallest analysis unit of the simultaneous interpretation delay detection method. By pairing and analyzing the first sentence frame and the second sentence frame, the detection granularity can be further refined to the frame level, which is beneficial to greatly improve the accuracy of detecting simultaneous interpretation delay.

[0091] In some disclosed embodiments, the latency measurement sub-module includes a frame mapping acquisition unit for obtaining a first frame number of a first statement and a second frame number of a second statement, and then obtaining a frame mapping relationship based on the ratio of the second frame number to the first frame number.

[0092] Therefore, the above solution points out a pairing method for statement frame pairs. The statement frame pair is the smallest analysis unit of the simultaneous translation latency detection method. By pairing and analyzing the first statement frame and the second statement frame, the detection granularity can be further refined to the frame level, which is conducive to greatly improving the accuracy of the simultaneous translation latency detection.

[0093] In some disclosed embodiments, the simultaneous translation latency detection device 40 detects the cumulative latency. The detection module 41 is used to obtain the cumulative latency of each statement pair based on the difference between the second frame number of the second statement and the first frame number of the first statement. The statistics module 42 is used to statistically obtain the cumulative latency of the simultaneous translation of the passage based on the cumulative latency of each statement pair.

[0094] In some disclosed embodiments, the simultaneous translation latency detection device 40 is used to detect the start-of-sentence latency. Similarly, the detection module 41 is used to obtain the start-of-sentence latency of each statement pair based on the time difference between the output of the first frame of the second statement and the input of the first frame of the first statement. The statistics module 42 is used to statistically obtain the start-of-sentence latency of the simultaneous translation of the passage based on the start-of-sentence latency of each statement pair.

[0095] In some disclosed embodiments, the simultaneous translation latency detection device 40 is used to detect the end-of-sentence latency. Similarly, the detection module 41 is used to obtain the end-of-sentence latency of each statement pair based on the time difference between the output of the last frame of the second statement and the input of the last frame of the first statement. The statistics module 42 is used to statistically obtain the end-of-sentence latency of the simultaneous translation of the passage based on the end-of-sentence latency of each statement pair.

[0096] Therefore, the above solution can detect various latencies during the simultaneous translation process, increasing the breadth of the simultaneous translation latency detection method, making the detection method more comprehensive, and helping to conduct a more detailed analysis of the simultaneous translation latency. In addition, since there is no need for manual latency statistics, the machine can automatically detect the simultaneous translation latency during the simultaneous translation process, which helps to significantly reduce the detection time and detection cost compared with manual detection.

[0097] Please refer to Figure 5 , Figure 5 which is a schematic framework diagram of an embodiment of the electronic device 50 of the present application. Specifically, the electronic device 50 includes a processor 51 and a memory 52 that are coupled to each other. The processor 51 is used to execute the program instructions stored in the memory 52 to implement the steps in the simultaneous translation latency detection method of any of the above embodiments. The electronic device 50 may specifically include, but is not limited to: a microcomputer, a notebook, a tablet computer, a smart phone, a simultaneous translation system, etc., which are not limited herein.

[0098] Specifically, the processor 51 can control itself and the memory 52 to execute the steps in the simultaneous translation delay detection method of any of the above embodiments. The processor 51 can also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip with signal processing capabilities. The processor 51 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. Additionally, the processor 51 can be implemented jointly by multiple integrated circuit chips.

[0099] In the above solution, the simultaneous translation delay of each sentence pair in the simultaneous translation process is detected, and the sentence pair includes the first sentence in the source language and the second sentence in the target language. The simultaneous translation delay of the sentence pair includes the frame-level delay of the sentence pair. Then, based on the simultaneous translation delay, the passage-level simultaneous translation delay is statistically obtained. On the one hand, in the simultaneous translation process, for the detection of passage-level delay, by first detecting the simultaneous translation delay at the sentence level, the detection granularity of the simultaneous translation delay can be refined. Moreover, the simultaneous translation delay at the sentence level further includes the frame-level delay, so that the detection granularity can be further refined to the frame level, which is conducive to greatly improving the accuracy of the simultaneous translation delay detection. On the other hand, since there is no need for manual delay statistics, the machine can automatically detect the simultaneous translation delay during the simultaneous translation process, which helps to significantly reduce the detection time and detection cost compared with manual detection.

[0100] Please refer to Figure 6 , Figure 6 FIG. is a schematic framework diagram of an embodiment of the computer-readable storage medium 60 of the present application. In this embodiment, the computer-readable storage medium 60 stores program instructions 601 that can be run by a processor, and the program instructions 601 are used to execute the steps in the embodiment of the above simultaneous translation delay detection method.

[0101] Specifically, the computer-readable storage medium 60 may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disc, or other media that can store program instructions. Alternatively, it may be a server storing the program instructions, which can send the stored program instructions to other devices for running, or can also run the stored program instructions by itself.

[0102] In the above solution, the simultaneous translation delay of each sentence pair in the simultaneous translation process is detected. The sentence pair includes the first sentence in the source language and the second sentence in the target language. The simultaneous translation delay of the sentence pair includes the frame-level delay of the sentence pair. Then, based on the simultaneous translation delay, the simultaneous translation delay of the passage is statistically obtained. On the one hand, in the simultaneous translation process, for the detection of the passage-level delay, by first detecting the simultaneous translation delay at the sentence level, the detection granularity of the simultaneous translation delay can be refined. Moreover, the simultaneous translation delay at the sentence level further includes the frame-level delay, so that the detection granularity can be further refined to the frame level, which is conducive to greatly improving the accuracy of the simultaneous translation delay detection. On the other hand, since there is no need for manual statistics of the delay, the machine can automatically detect the simultaneous translation delay during the simultaneous translation process, which helps to significantly reduce the detection time and detection cost compared with manual detection.

[0103] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical, or other form.

[0104] The unit described as a separated component may or may not be physically separated. The component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] In addition, each functional unit in various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0106] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods according to various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

Claims

1. A simultaneous interpretation delay detection method, characterized in that Including: Detecting the simultaneous interpretation delay of each sentence pair during the simultaneous interpretation process; wherein, the sentence pair includes a first sentence in the source language and a second sentence in the target language, and the simultaneous interpretation delay of the sentence pair includes the frame-level delay of the sentence pair, and the frame-level delay is obtained based on the delay time of each sentence frame pair in the sentence pair. The sentence frame pair includes a first sentence frame in the first sentence and a second sentence frame in the second sentence. The first sentence frame is the sentence frame numbered with the first ordinal number in the first sentence, and the second sentence frame is the sentence frame numbered with the second ordinal number in the second sentence. The first ordinal number and the second ordinal number satisfy a frame mapping relationship; Based on the simultaneous interpretation delay of the sentence, the text simultaneous interpretation delay is statistically obtained.

2. The method according to claim 1, wherein The steps for obtaining the frame-level delay include: Obtaining the sentence judgment result of the sentence pair; wherein, the sentence judgment result includes whether the sentence pair is the first pair of simultaneous interpretation sentences in the simultaneous interpretation process; 3. The method according to claim 2, wherein Based on the sentence judgment result of the sentence pair and the delay time of each sentence frame pair in the sentence pair, the frame-level delay of the sentence pair is obtained. The obtaining the frame-level delay of the sentence pair based on the sentence judgment result of the sentence pair and the delay time of each sentence frame pair in the sentence pair includes: Based on the sentence judgment result, obtaining the influence factor of the sentence pair; wherein, the influence factor represents the influence degree of the reference sentence pair of the sentence pair on the simultaneous interpretation delay caused by the sentence pair; 4. The method according to claim 3, wherein Based on the delay time of each sentence frame pair in the sentence pair and the influence factor, the frame-level delay of the sentence pair is obtained. The obtaining the influence factor of the sentence pair based on the sentence judgment result includes: In response to the sentence judgment result including that the sentence pair is the first pair of simultaneous interpretation sentences, setting the influence factor to a preset value; 5. The method according to claim 3, wherein And / or, in response to the sentence judgment result including that the sentence pair is not the first pair of simultaneous interpretation sentences, obtaining the influence factor based on the end delay of the reference sentence pair and the start delay of the sentence pair. The influence factor includes the influence duration of the reference sentence pair on the simultaneous interpretation delay caused by the sentence pair; The obtaining the frame-level delay of the sentence pair based on the delay time of each sentence frame pair in the sentence pair and the influence factor includes: Statistically obtaining the cumulative sum of the delay time of the sentence frame pair to obtain the overall delay in the simultaneous interpretation process; 6. The method according to claim 1, wherein Based on the ratio of the overall delay to the number of sentence frame pairs and the influence factor, the frame-level delay is obtained. The steps for obtaining the sentence frame pair include: Based on the frame mapping relationship and the first ordinal number of the first sentence frame in the first sentence, determining the second ordinal number; 7. The method according to claim 6, wherein Combining the first sentence frame with the first ordinal number and the second sentence frame with the second ordinal number to form the sentence frame pair. The steps for obtaining the frame mapping relationship include: Obtaining the first number of frames of the first sentence and the second number of frames of the second sentence; Based on the ratio of the second number of frames to the first number of frames, obtaining the frame mapping relationship.

8. The method according to claim 1, characterized in that The simultaneous interpretation delay of the statement further includes an accumulated delay, which is obtained based on the difference between the second number of frames of the second statement and the first number of frames of the first statement.

9. A simultaneous interpretation delay detection device, characterized in that, Comprising: a detection module, configured to detect the simultaneous interpretation delay of each statement pair in the simultaneous interpretation process, where the statement pair includes a first statement in the source language and a second statement in the target language, and the simultaneous interpretation delay of the statement pair includes the frame-level delay of the statement pair, and the frame-level delay is obtained based on the delay time of each statement frame pair in the statement pair, where the statement frame pair includes a first statement frame in the first statement and a second statement frame in the second statement, the first statement frame is the statement frame numbered with the first ordinal number in the first statement, the second statement frame is the statement frame numbered with the second ordinal number in the second statement, and the first ordinal number and the second ordinal number satisfy a frame mapping relationship; a statistics module, configured to statistically obtain the simultaneous interpretation delay of the passage based on the simultaneous interpretation delay of the statement.

10. An electronic device, characterized in that, Comprising a processor and a memory coupled to each other; the processor is configured to execute the program instructions stored in the memory to implement the simultaneous interpretation delay detection method according to any one of claims 1-8.

11. A storage medium, characterized in that, Stored with program instructions that can be run by a processor, and the program instructions are used to implement the simultaneous interpretation delay detection method according to any one of claims 1-8.

Images