Method for detecting language direction misselection of translation device and translation device

Abstract

The application discloses a language direction misselection detection method of a translation device and the translation device, and relates to the technical field of machine translation. The method comprises the following steps: acquiring input speech when the translation device is in a limited translation mode, executing a limited recognition process and an open language verification process, wherein the limited recognition process performs language recognition according to a limited language set, and the open language verification process performs language recognition without being restricted by the limited language set to obtain language recognition confidence information of an open candidate language; and determining whether the input speech has a language direction misselection conflict according to the output of the limited recognition process and the output of the open language verification process. The application can actively discover and diagnose the mismatch between the language direction configuration and the actual input when the device is in the limited translation mode, thereby providing a solid technical foundation for solving translation errors caused by language direction misselection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine translation, and more particularly, to a translation device language direction misselection detection method and a translation device. BACKGROUND

[0002] A translation device usually needs to complete the processing procedures such as speech collection, language recognition, speech transcription and machine translation between different language users. In order to adapt to diversified use scenarios and reduce the operation threshold, the existing translation device provides a plurality of language direction configuration modes.

[0003] In an earlier technical solution, a user needs to pre-select two specific languages and bind different translation directions through physical or virtual keys, and the device determines the source language and the target language according to the key operation of the user. With the development of technology, a solution supporting bilingual automatic determination appears, and the user only needs to select two candidate languages, and the device can automatically determine which language the input speech belongs to and perform translation under the triggering of one key. In addition, some latest generation devices also support a multi-language automatic determination mode, which allows the system to automatically identify and translate a plurality of supported languages without fixed languages.

[0004] However, in the prior art, when the translation device is in a bilingual automatic determination or key binding limited translation mode, if the language of the actual input speech does not belong to the limited language set selected by the user, the device often forces the language classification and recognition within the limited set. This forced classification can cause serious errors in speech recognition text, and further produce incorrect translation results, so that the user perceives that the translation is inaccurate, but it is difficult to detect that the root cause is that the language direction configuration is inconsistent with the actual input language, that is, there is a language direction misselection conflict in the actual input speech. SUMMARY

[0005] In view of the above problems, the present application is proposed to provide a translation device language direction misselection detection method and a translation device to accurately identify whether there is a language direction misselection conflict in the input speech when the translation device is in a limited translation mode. The specific scheme is as follows:

[0006] In a first aspect, a translation device language direction misselection detection method is provided, comprising:

[0007] acquiring input speech when the translation device is currently in a limited translation mode, wherein the translation device is only allowed to perform language recognition on the input speech within a limited language set in the limited translation mode;

[0008] The process involves executing a restricted recognition process and an open language verification process. The restricted recognition process involves recognizing the language of the input speech according to the restricted language set. The open language verification process involves recognizing the language of the input speech without being constrained by the restricted language set, in order to obtain language recognition confidence information for open candidate languages. The open candidate languages ​​include candidate languages ​​within the restricted language set and candidate languages ​​outside the restricted language set.

[0009] Based on the output of the restricted recognition process and the output of the open language verification process, it is determined whether the input speech has a speech direction misselection conflict.

[0010] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of determining whether the input speech has a speech direction misselection conflict based on the output of the restricted recognition process and the output of the open language verification process includes:

[0011] Based on the output of the restricted recognition process and the output of the open language verification process, a conflict index is calculated. The conflict index is used to measure the degree of conflict between the input speech and the current speech direction configuration.

[0012] Based on the conflict index, it is determined whether the input speech has a speech direction misselection conflict.

[0013] In one possible design, in another implementation of the first aspect of the embodiments of this application, the conflict index includes: an index of language identification confidence dimension;

[0014] The process of calculating the conflict index of the speech recognition confidence dimension includes:

[0015] Based on the language identification confidence information of the open candidate languages ​​output by the open language verification process, the first candidate language with the highest confidence and its confidence are determined among the candidate languages ​​within the set, and the first candidate language outside the set with the highest confidence and its confidence are determined among the candidate languages ​​outside the set.

[0016] The conflict index of the speech recognition confidence dimension is determined based on the confidence scores of candidate languages ​​within the first set and the confidence scores of candidate languages ​​outside the first set.

[0017] In one possible design, in another implementation of the first aspect of the embodiments of this application, the conflict index of the speech recognition confidence dimension includes:

[0018] The difference or ratio between the confidence scores of candidate languages ​​within the first set and the confidence scores of candidate languages ​​outside the first set.

[0019] In one possible design, in another implementation of the first aspect of the embodiments of this application, the output of the restricted recognition process includes: restricted recognition language, first ASR text obtained after recognizing the input speech according to the restricted recognition language, and recognition confidence;

[0020] The output of the open language verification process includes: open candidate languages ​​and their corresponding language recognition confidence scores, and the second ASR text obtained after recognizing the input speech using the open candidate language with the highest confidence score, along with its recognition confidence score.

[0021] The conflict index also includes an index for ASR recognition confidence dimension, which includes: recognition confidence of the first ASR text, recognition confidence of the second ASR text, and / or, the difference in recognition confidence between the first ASR text and the second ASR text.

[0022] In one possible design, in another implementation of the first aspect of the embodiments of this application, the output of the restricted recognition process includes: a restricted recognition language, a first ASR text obtained by recognizing the input speech according to the restricted recognition language, a first translated text after translating the first ASR text and its quality score;

[0023] The output of the open language verification process includes: open candidate languages ​​and their corresponding language recognition confidence scores, the second ASR text obtained after recognizing the input speech using the open candidate language with the highest confidence score, and the second translated text and its quality score after translating the second ASR text.

[0024] The conflict index also includes indicators for the quality dimension of the translated text, which include: the quality score of the first translated text, the quality score of the second translated text, and / or the difference in quality scores between the first and second translated texts.

[0025] In one possible design, in another implementation of the first aspect of the embodiments of this application, the conflict index further includes:

[0026] The metrics for input speech quality include at least one of the following: input speech duration, input speech signal-to-noise ratio, and metrics characterizing the stability of input speech VAD detection results.

[0027] In one possible design, in another implementation of the first aspect of the embodiments of this application, the conflict index further includes:

[0028] The consecutive rounds of the candidate languages ​​outside the first set determined in the current translation round, wherein the consecutive rounds represent the number of times the candidate languages ​​outside the first set determined in the current translation round have been consecutively used as candidate languages ​​outside the first set in the current round and before the current round.

[0029] In one possible design, in another implementation of the first aspect of the present application, the output of the restricted recognition process includes: a restricted recognition language, a first ASR text obtained by recognizing the input speech according to the restricted recognition language and the recognition confidence level, and a first translated text and its quality score after translating the first ASR text.

[0030] The output of the open language verification process includes: open candidate languages ​​and their corresponding language recognition confidence scores, the second ASR text obtained after recognizing the input speech using the open candidate language with the highest confidence score and its recognition confidence score, and the second translated text and its quality score after translating the second ASR text.

[0031] The process of calculating the conflict index based on the output of the restricted identification process and the output of the open language verification process includes:

[0032] Based on the language identification confidence information of the open candidate languages ​​output by the open language verification process, the first candidate language with the highest confidence and its confidence are determined among the candidate languages ​​within the set, and the first candidate language outside the set with the highest confidence and its confidence are determined among the candidate languages ​​outside the set.

[0033] Calculate the difference and / or ratio between the confidence scores of candidate languages ​​within the first set and the confidence scores of candidate languages ​​outside the first set;

[0034] Calculate the quality difference between the second translated text quality score and the first translated text quality score;

[0035] Obtain the consecutive rounds of the first set of candidate languages ​​determined in the current translation round, wherein the consecutive rounds represent the number of times the first set of candidate languages ​​determined in the current translation round has been consecutively used as the first set of candidate languages ​​in the current round and before the current round;

[0036] Calculate the quality score of the input speech;

[0037] The confidence scores, differences and / or ratios, quality score differences, consecutive rounds, and quality scores of the input speech of candidate languages ​​outside the first set are weighted and fused to obtain a comprehensive conflict score, which is used as the conflict index.

[0038] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of determining whether the input speech has a speech direction misselection conflict based on the conflict index includes:

[0039] According to the configured conflict level classification rules, the speech direction misselection conflict level corresponding to the input speech is determined based on the conflict index. The speech direction misselection conflict level includes at least three levels, one of which represents that there is no speech direction misselection conflict, and the remaining levels represent different degrees of speech direction misselection conflict.

[0040] In one possible design, another implementation of the first aspect of the embodiments of this application further includes:

[0041] Based on the speech direction misselection conflict level corresponding to the input speech, the translation result and / or prompt information are output using an output strategy that matches the speech direction misselection conflict level.

[0042] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of outputting the translation result and / or prompt information using an output strategy that matches the level of speech misselection conflict includes:

[0043] When there is no misdirection conflict in the described symphony misselection conflict level representation, the translation result is output according to the restricted identification process.

[0044] When the misselection conflict level of the speech direction represents a low-confidence conflict level, the translation result is output according to the restricted identification process, and the conflict index calculated in the current translation round is recorded in the background.

[0045] When the conflict level is believed in the speech direction misselection conflict level representation, the translation result is output according to the restricted identification process, and a non-blocking prompt strategy is executed. The non-blocking prompt strategy is to output speech direction misselection prompts to the user while maintaining the normal output of the translation result of the restricted identification process, and the prompts do not block the current translation process.

[0046] When the misselection conflict level represents a high-confidence conflict level, perform one or more of the following processes:

[0047] The user is prompted to switch their source language to the recommended language, which is the open candidate language with the highest confidence level.

[0048] Simultaneously output the translation results of the restricted recognition process and the translation results of the open language verification process;

[0049] Block the translation results output by the restricted recognition process;

[0050] Incorrect use of the voice selection button.

[0051] In one possible design, in another implementation of the first aspect of the embodiments of this application, the open candidate languages ​​include all languages ​​supported by the translation device;

[0052] Alternatively, the open candidate languages ​​are a set of language seeds selected from all languages ​​supported by the translation device according to preset filtering rules, and the open candidate languages ​​include at least one candidate language that does not belong to the restricted language set.

[0053] In one possible design, in another implementation of the first aspect of the embodiments of this application, the preset filtering rules include at least one of the following: geographical information, historical conversation information, user's commonly used language information, contact information, and system resource information.

[0054] Secondly, a translation device is provided, including: a memory and a processor;

[0055] The memory is used to store programs;

[0056] The processor is configured to execute the program to implement each step of the speech direction misselection detection method for the translation device described in any of the first aspects above.

[0057] Thirdly, a readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the speech direction misselection detection method of the translation device described in any of the first aspects.

[0058] Fourthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the speech direction misselection detection method for the translation device described in any of the first aspects.

[0059] By employing the aforementioned technical solution, this application achieves accurate detection of speech direction misselection even when the translation device is in restricted translation mode. By introducing an open language bypass verification process outside the restricted recognition main process, the device can acquire the confidence distribution of the input speech in the open candidate language space in real time without interfering with the user's normal operating habits. The output of the restricted recognition process reflects the recognition result under the user's intent configuration, while the output of the open language verification process reveals the true language tendency of the speech itself. By comparing these two types of outputs, the system can determine whether there is a speech direction misselection conflict in the input speech.

[0060] This application's solution, when determining pre-selected conflicts, simultaneously considers the outputs of both the restricted recognition process and the open language verification process, relying on the relative correlation between the two types of outputs to complete the judgment, rather than solely relying on a fixed absolute threshold. This avoids the false alarms that are easily generated by a purely open language recognition mode, and also solves the inherent deficiency of traditional bilingual automatic judgment and button binding solutions in their inability to recognize and perceive a third language. This application's solution enables translation devices to proactively detect and diagnose mismatches between the language direction configuration and the actual input, providing a solid technical foundation for solving translation errors caused by incorrect language direction selection. Attached Figure Description

[0061] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0062] Figure 1a This example illustrates a flowchart of a translation method in a key-binding contextual mode.

[0063] Figure 1b This example illustrates a flowchart of a translation method in an automatic bilingual determination mode.

[0064] Figure 2 A schematic diagram of an implementation system architecture for the speech direction misselection detection method provided in this application embodiment;

[0065] Figure 3 This is a schematic flowchart of a speech direction misselection detection method provided in an embodiment of this application;

[0066] Figure 4 A schematic diagram of another speech direction misselection detection method provided in this application embodiment;

[0067] Figure 5 This application provides a schematic diagram of an output strategy corresponding to different conflict levels.

[0068] Figure 6 This is a schematic diagram of the structure of a translation device provided in an embodiment of this application. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0071] In existing translation device applications, users typically need to activate the translation function by binding a key to the language direction or by automatically determining bilingual meaning. (Refer to...) Figure 1a and Figure 1b As shown.

[0072] In dual physical button or virtual button language binding mode, users can select two specific languages, such as Chinese + English. They can set the translation language for button A to translate from Chinese to English, and the translation language for button B to translate from English to Chinese. The system forcibly determines the source and target languages ​​for the input speech based on the currently pressed button. For example, when the user presses button A, the system forcibly determines that the source language for the input speech is Chinese and the target language is English.

[0073] In bilingual auto-determination mode, the user selects two candidate languages, such as Chinese + English. The system provides a physical or virtual button as the recording input. After acquiring the input speech, the system selects only one of the two candidate languages ​​as the source language and the other as the target language. It does not detect any third language other than the two candidate languages.

[0074] Both key-based language binding and automatic bilingual identification fall under the category of restricted translation mode. Restricted translation mode refers to a working state in which the translation device limits the language recognition range to a specific subset (a restricted set of languages) based on the user's current explicit selection or implicit configuration.

[0075] In these restricted translation modes, devices are forced to perform speech recognition and translation within a limited set of languages ​​selected by the user (e.g., Chinese and English). However, when the actual input speech is not in this limited language set (e.g., the other party is actually speaking Russian, but the device is in Chinese-English bilingual mode), existing devices often force the speech to be classified as a language within the limited set (e.g., incorrectly identifying it as English), resulting in garbled text and severely distorted translation results. Users usually only perceive the inaccurate translation but are unlikely to realize that the root cause lies in the inconsistency between the speech direction configuration and the actual input language.

[0076] Some existing solutions offer an automatic multilingual translation mode, allowing the system to automatically identify and translate multiple supported languages ​​without being restricted to a single language.

[0077] However, existing technologies lack an effective mechanism to proactively detect intrusion conflicts from third languages ​​outside the restricted language set when the translation device is in restricted translation mode.

[0078] In view of this, this application provides a method for detecting speech direction mismatch in a translation device, which can detect the mismatch between speech direction configuration and the actual input language when the translation device is in a restricted translation mode, that is, whether there is a speech direction mismatch conflict in the input speech.

[0079] The speech misselection detection method of this application can be applied to, for example... Figure 2 The system architecture shown may include a terminal 100 and a server 200. The server 200 may include one or more servers (…). Figure 2 (This example uses a server as an illustration).

[0080] Either terminal 100 or server 200 can be used independently to execute the speech direction misselection detection method provided in the embodiments of this application. Alternatively, terminal 100 and server 200 can also be used collaboratively to execute the speech direction misselection detection method provided in the embodiments of this application.

[0081] The following description Figure 2 The product form of the mid-terminal 100;

[0082] The terminal 100 in this application embodiment can be a translator, voice recorder, mobile phone, tablet computer, teaching large screen, wearable device, vehicle-mounted device, conference terminal, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0083] This application provides a method for detecting speech direction misselection. Taking the application of this method to a computer device as an example, the computer device can specifically be... Figure 2 The system consists of terminal 100 or a combination of terminal 100 and server 200. (Refer to...) Figure 3 The specific steps of this speech direction misselection detection method are as follows:

[0084] Step S100: Acquire input speech while the translation device is currently in restricted translation mode. In restricted translation mode, the translation device is only allowed to perform language recognition on the input speech within a restricted language set.

[0085] In this context, restricted translation mode refers to a working state where the translation device limits the language recognition range to a specific subset based on the user's explicit selection or implicit configuration. The restricted language set is a list of languages ​​allowed for language recognition, determined by user operation. This list can originate from a bilingual combination manually selected by the user via physical buttons or a virtual interface, such as {Chinese, English}.

[0086] Input voice can be captured in real time via a microphone array or read from an audio file on a storage medium.

[0087] This step captures the original audio signal to be processed, while preserving the user's existing interaction habits, as the data basis for subsequent verification.

[0088] For example, when a user presses the physical button bound to Chinese to English, or clicks the Chinese-English translation option on the interface, the device enters the restricted translation mode, at which time the restricted language set is locked to {zh,en}.

[0089] Step S110: Execute the restricted recognition process and the open language verification process.

[0090] The restricted language recognition process involves identifying the language of the input speech according to a restricted language set. The open language verification process involves identifying the language of the input speech without being constrained by the aforementioned restricted language set, in order to obtain language recognition confidence information for open candidate languages. Open candidate languages ​​include candidate languages ​​within the restricted language set and candidate languages ​​outside the restricted language set.

[0091] The restricted recognition process uses the original main processing link of the translation device. Its function is to output recognition results that meet expectations within the framework configured by the user, thus ensuring the continuity of the business process.

[0092] The open language verification process is a parallel or serial bypass processing link. Its function is to break through the limitations of user configuration and explore the true language attribution of the input speech in the full language space or a predefined broad language seed set (open candidate language set).

[0093] In some possible implementations, the open candidate languages ​​can include all languages ​​supported by the translation device. Alternatively, the open candidate languages ​​can be a set of language seeds selected from all languages ​​supported by the translation device according to preset filtering rules, and the open candidate languages ​​must include at least one candidate language that does not belong to the restricted language set.

[0094] The preset filtering rules include at least one of the following: geographic information, historical conversation information, user's commonly used language information, contact information, system resource information, etc.

[0095] Preset filtering rules can refer to logical strategies used to narrow down the language recognition search space and reduce computational overhead. These rules can be dynamically generated based on multiple dimensions, such as filtering commonly used languages ​​in the current geographical location of the device, filtering frequently used languages ​​based on user history conversation records, filtering potential communication languages ​​based on contact information in the address book, or filtering a subset of languages ​​within the resource-allowed range based on the device's current battery status and computing load. By introducing preset filtering rules, full language recognition is transformed into subset recognition, significantly reducing the latency and power consumption of the open language verification process.

[0096] Candidate languages ​​within the set can refer to those languages ​​that appear in both the open candidate language set and the current restricted language set; candidate languages ​​outside the set can refer to those languages ​​that appear in the open candidate language set but do not belong to the restricted language set.

[0097] The output of a restricted language recognition process typically includes the restricted language recognition result (referred to as the restricted recognition language), the corresponding ASR text (defined as the first ASR text), and the recognition confidence level.

[0098] Alternatively, in the restricted recognition process, the first ASR text can be further translated to obtain the first translated text and its quality score.

[0099] The output of the open language verification process is a distribution list containing multiple open candidate languages ​​and their confidence scores. For example, when the restricted language set is {Chinese, English}, but the input speech is actually Russian, the restricted recognition process might force an output of English with a lower confidence score, while the open language verification process can output a complete distribution including Russian (0.91), English (0.04), and Chinese (0.02). The scores in parentheses represent the confidence scores for the corresponding languages.

[0100] like Figure 4 As shown, Figure 4 The diagram illustrates the parallel execution of the restricted recognition process and the open language verification process: the left branch represents the restricted recognition process, which is strictly calculated within the restricted language set R; the right branch represents the open language verification process, which is calculated within the open candidate language set U. Both processes work together on the same input speech. This dual-process design maintains the stability of the user interface through the restricted recognition process, while the open language verification process provides a global perspective for discovering third languages. Together, they enable the device to obtain the true confidence distribution reflecting the intrinsic attributes of the speech in restricted translation mode, providing comprehensive data support for determining whether there is a misselection of speech direction.

[0101] It should be noted that the restricted recognition process and the open language verification process are not limited to parallel execution; they can also be executed sequentially. Figure 4 Only one optional execution order is shown.

[0102] Optionally, the open language verification process can be implemented through an open language verification module, which can be deployed on local devices, cloud servers, or a hybrid local and cloud architecture. Local deployment can provide a quick, coarse assessment, while cloud deployment can provide a more precise verification.

[0103] Step S120: Based on the output of the restricted recognition process and the output of the open language verification process, determine whether there is a speech direction misselection conflict in the input speech.

[0104] The process of determining whether there is a speech direction misselection conflict in the input speech can be based on the difference between the output of the restricted recognition process and the output of the open language verification process. A speech direction misselection conflict refers to a discrepancy between the user's currently configured set of restricted languages ​​and the actual language of the input speech.

[0105] This step utilizes the global perspective of the open language verification process to identify local deviations in the restricted recognition process, thereby drawing conclusions about whether the current translation direction configuration is correct. As a result, the device can accurately identify that the root cause of inaccurate translation is an incorrect direction configuration, rather than simply insufficient recognition model performance, providing a reliable decision-making basis for subsequently implementing different output strategies.

[0106] In some possible implementations, the result of determining whether the input speech has a speech direction misselection conflict in this step can be a binary classification result, i.e., present or absent. Alternatively, it can be a multi-class classification result, i.e., determining the speech direction misselection conflict level corresponding to the input speech according to the configured conflict level classification rules. This speech direction misselection conflict level includes at least three levels, one of which represents the absence of speech direction misselection conflict, and the remaining levels represent different degrees of speech direction misselection conflict.

[0107] Alternatively, the translation result and / or prompt information can be output using an output strategy that matches the level of speech misselection conflict corresponding to the input speech.

[0108] The method provided in this application enables accurate detection of speech direction misselection when the translation device is in restricted translation mode. By introducing an open language bypass verification process outside the restricted recognition main process, the device can acquire the confidence distribution of the input speech in the open candidate language space in real time without interfering with the user's normal operating habits. The output of the restricted recognition process reflects the recognition result under the user's intention configuration, while the output of the open language verification process reveals the true language tendency of the speech itself. By comparing these two types of outputs, the system can determine whether there is a speech direction misselection conflict in the input speech.

[0109] This application's solution, when determining pre-selected conflicts, simultaneously considers the outputs of both the restricted recognition process and the open language verification process, relying on the relative correlation between the two types of outputs to complete the judgment, rather than solely relying on a fixed absolute threshold. This avoids the false alarms that are easily generated by a purely open language recognition mode, and also solves the inherent deficiency of traditional bilingual automatic judgment and button binding solutions in their inability to recognize and perceive a third language. This application's solution enables translation devices to proactively detect and diagnose mismatches between the language direction configuration and the actual input, providing a solid technical foundation for solving translation errors caused by incorrect language direction selection.

[0110] In some embodiments of this application, an optional implementation of the aforementioned step S120 is described, specifically:

[0111] First, a conflict index is calculated based on the output of the restricted recognition process and the output of the open language verification process.

[0112] The conflict index measures the degree of conflict between the input speech and the current speech direction configuration. It is a quantitative value constructed based on the restricted result data generated by the restricted recognition process and the data generated by the open language verification process.

[0113] Reference Figure 4 As shown, the output of the restricted recognition process typically includes the source language (restricted recognition language) determined within the restricted language set, the speech recognition text (first ASR text) after recognizing the input speech according to the restricted recognition language and its recognition confidence, and the translation result (first translated text) generated based on the first ASR text and its quality score.

[0114] The output of the open language verification process may include multiple open candidate languages ​​not constrained by the restricted language set, along with their corresponding language recognition confidence information. These open candidate languages ​​include both candidate languages ​​within the restricted language set and candidate languages ​​outside the restricted language set. Further optionally, the output of the open language verification process may also include: a second ASR text obtained by recognizing the input speech using the open candidate language with the highest confidence level, along with its recognition confidence level, and a translation result (second translated text) generated based on this second ASR text, along with its quality score.

[0115] The process of calculating the conflict index can involve extracting key feature parameters from the two types of outputs mentioned above and performing a fusion operation through a preset mathematical model or rule logic to generate a parameter that can characterize the degree of deviation between the user's current speech orientation configuration and the actual attributes of the input speech.

[0116] Furthermore, based on the calculated conflict index, it is determined whether there is a speech direction misselection conflict in the input speech.

[0117] This step can perform threshold judgment or level mapping logic based on the conflict index calculated in the previous step to output a clear conflict determination result.

[0118] Specifically, the system can compare the calculated conflict index with a preset conflict determination rule base, which stores threshold ranges or decision boundaries configured for different scenarios.

[0119] This application introduces a conflict index as an intermediate quantitative object to organically integrate the output data of the restricted recognition process and the open language verification process. A conflict index is constructed that can objectively reflect the degree of matching between the speech direction configuration and the actual speech; then, based on this conflict index, it is determined whether there is a speech direction misselection conflict in the input speech.

[0120] Reference Figure 4 As shown, conflict indicators can include various forms, including but not limited to: highest confidence within the set, highest confidence outside the set, confidence difference, confidence ratio, ASR text quality difference, translation quality difference, number of consecutive conflicts, input speech quality, etc.

[0121] In this embodiment of the application, the conflict index can be divided into multiple different dimensions of indexes, including but not limited to:

[0122] 1. Indicators for language identification confidence level.

[0123] The process of calculating the conflict index for the confidence dimension of speech recognition can include:

[0124] S1. Based on the language identification confidence information of the open candidate languages ​​output by the open language verification process, determine the first candidate language with the highest confidence in the set and its confidence among the candidate languages ​​within the set, and determine the first candidate language outside the set with the highest confidence and its confidence among the candidate languages ​​outside the set.

[0125] Among them, the language recognition confidence information of open candidate languages ​​can refer to the posterior probability or score value of each open candidate language belonging to the input speech after the open language verification process performs language recognition within the open candidate language set on the input speech.

[0126] Let R be the set of restricted languages, L be the open candidate languages ​​output by the open language verification process, and P_open(l) be the language recognition confidence score of language l in the output of the open language verification process. Then we can calculate:

[0127] L_in_best=argmax_{l in R} P_open(l);

[0128] L_out_best=argmax_{l not in R} P_open(l);

[0129] P_open(L_out_best);

[0130] P_open(L_in_best);

[0131] Wherein, L_in_best represents the language with the highest confidence level in open language identification within the restricted language set R, which is also the candidate language within the first set mentioned above, and P_open(L_in_best) represents the confidence level of the candidate language within the first set. L_out_best represents the language with the highest confidence level in open language identification outside the restricted language set R, which is also the candidate language outside the first set mentioned above, and P_open(L_out_best) represents the confidence level of the candidate language outside the first set.

[0132] S2. Based on the confidence levels of candidate languages ​​within the first set and the confidence levels of candidate languages ​​outside the first set, determine the conflict index for the confidence dimension of speech recognition.

[0133] Specifically, the confidence scores of candidate languages ​​within the first set and the confidence scores of candidate languages ​​outside the first set can be used as conflict indicators for the confidence dimension of speech recognition.

[0134] Furthermore, it can be calculated that:

[0135] D=P_open(L_out_best)-P_open(L_in_best);

[0136] Q=P_open(L_out_best) / max(P_open(L_in_best),epsilon).

[0137] Where D represents the confidence difference between candidate languages ​​outside the first set and candidate languages ​​inside the first set, Q represents the confidence ratio between candidate languages ​​outside the first set and candidate languages ​​inside the first set, and epsilon is used to avoid the minimum value where the denominator is zero.

[0138] The aforementioned indicators D and Q can also be used as conflict indicators for the confidence dimension of speech recognition.

[0139] In summary, the conflict indicators for the confidence dimension of speech recognition include, but are not limited to: the confidence of candidate languages ​​within the first set ( Figure 4 The confidence scores of candidate languages ​​outside the first set are referred to as the highest confidence score within the set and the confidence scores of candidate languages ​​outside the first set. Figure 4 (abbreviated as the highest confidence level outside the set), D ( Figure 4 The difference is referred to as confidence level difference (Q). Figure 4 This is referred to as the confidence ratio.

[0140] This application embodiment obtains the confidence distribution of open candidate languages ​​through an open language verification process, and accurately extracts the optimal candidate languages ​​(candidate languages ​​within the first set and candidate languages ​​outside the first set) and their confidence scores from within and outside the restricted set. This achieves the perception of potential speech direction misselection risks while preserving the user's original restricted translation mode. The candidate languages ​​within the first set represent the languages ​​that the system considers most likely under the current configuration, while the candidate languages ​​outside the first set represent the real languages ​​to which the actual input speech is most likely to belong. The comparison of their confidence scores directly reveals the degree of deviation between the configuration and reality. By determining the conflict index of the speech recognition confidence dimension based on the confidence scores of the optimal candidate languages ​​within and outside the restricted set, a stable and reliable quantitative basis is provided for identifying speech direction misselection conflicts.

[0141] 2. Indicators for ASR identification confidence dimension.

[0142] Specifically, the output of the restricted recognition process includes: the restricted recognition language, the first ASR text obtained after recognizing the input speech in the restricted recognition language, and the recognition confidence level.

[0143] The output of the open language verification process includes: open candidate languages ​​and their corresponding language recognition confidence scores, and the second ASR text obtained after recognizing the input speech using the open candidate language with the highest confidence score, along with its recognition confidence score.

[0144] Based on this, the conflict index can also include an index of ASR recognition confidence dimension, which may include: the recognition confidence of the first ASR text, the recognition confidence of the second ASR text, and / or, the difference in recognition confidence between the first ASR text and the second ASR text. Figure 4 This is referred to as ASR (Aspect Rating Confidence Difference) in Chinese.

[0145] This embodiment introduces a conflict index within the ASR recognition confidence dimension, extending the perspective of speech direction misselection detection from a simple language probability distribution to speech recognition quality feedback. The first ASR text output by the restricted recognition process, along with its low confidence, reveals a failure in speech understanding under the current speech direction configuration; while the second ASR text output by the open language verification process, with its high confidence, confirms the existence of a superior language. When there is a significant difference between these two confidence indices, the system can further confirm that a speech direction misselection has occurred, rather than merely a probability fluctuation caused by background noise or short speech segments. Based on this, combined with the aforementioned language recognition confidence dimension index, the system can construct a multi-dimensional conflict evaluation system, thereby more accurately identifying speech direction misselection conflicts or classifying conflict levels.

[0146] 3. Indicators for the quality dimension of translated texts.

[0147] Specifically, the output of the restricted recognition process may include: the restricted recognition language, the first ASR text obtained after recognizing the input speech in the restricted recognition language, the first translated text after translating the first ASR text, and its quality score.

[0148] The output of the open language verification process may include: open candidate languages ​​and their corresponding language recognition confidence scores, the second ASR text obtained after recognizing the input speech using the open candidate language with the highest confidence score, and the second translated text and its quality score after translating the second ASR text.

[0149] Building upon this, conflict indicators can also include metrics related to the quality dimension of the translated text, which may include: the quality score of the first translated text, the quality score of the second translated text, and / or the difference in quality scores between the first and second translated texts. Figure 4 (This is referred to as the difference in translation quality in Chinese).

[0150] This embodiment further incorporates the quality dimension of the translated text into the conflict indicator system. Based on this, the difference in quality scores between the first and second translated texts intuitively characterizes whether the current language direction configuration has led to a substantial deterioration in the translation result. By comparing the results of the restricted identification process and the open language verification process, not only can inconsistencies in language be detected at the acoustic identification stage, but the final damage caused by incorrect language direction selection can also be confirmed at the translation generation stage. This end-to-end verification mechanism from the identification end to the translation end makes the determination of conflict indicators more closely aligned with the user's actual perception experience. When the translation quality dimension indicators show a significant discrepancy, the system can be certain that the current restricted translation result is unusable, thereby more accurately identifying language direction selection conflicts or classifying conflict levels.

[0151] 4. Input speech quality metrics.

[0152] The metrics for input speech quality may include at least one of the following:

[0153] Input speech duration, input speech signal-to-noise ratio, and indicators characterizing the stability of input speech activity detection (VAD) results, etc.

[0154] The input speech duration refers to the duration of the valid speech signal acquired in the current round. It can be calculated as the time difference obtained by the Voice Activity Detection (VAD) module after determining the start and end points of the audio stream. This metric measures whether the input speech contains sufficient acoustic features for the language recognition model to make a reliable judgment. For example, when the input speech duration is less than 500 milliseconds, it usually means that the user only uttered very short interjections such as "um" or "okay." Such short speech lacks a complete phoneme sequence, which can easily lead to random high-confidence misjudgments in the open language verification process. By introducing the speech duration metric, the system can automatically reduce the sensitivity of subsequent conflict judgments when the speech duration is too short, preventing misjudgments due to incorrect speech direction selection caused by insufficient data.

[0155] The input speech signal-to-noise ratio (SNR) refers to the ratio of effective speech signal energy to ambient noise energy. It is calculated by segmenting the acquired audio signal into frames and estimating the power of each signal frame and the power of the noise frame. This metric characterizes the noise level of the current recording environment and the purity of the speech signal. For example, in noisy environments such as subway stations or busy city streets, if the input speech SNR is low, background noise may mask the language-specific spectral characteristics, leading to a distortion in the confidence distribution output by the language recognition model. In this case, even if the open language verification process outputs a high confidence score for out-of-set candidate languages, the system will discount the reliability of that confidence score based on the low SNR metric, thereby avoiding misjudgments and conflicts in speech direction selection under harsh acoustic conditions.

[0156] An indicator characterizing the stability of input speech VAD detection results can be a quantitative value reflecting the frequency of changes in the speech activity detection state over time. This can be determined based on the number of times the VAD state transitions from spoken to silent or vice versa per unit time, or the proportion of breaks in effective speech segments. This indicator is used to identify unstable scenarios such as intermittent speech, speech truncation due to strong interference, or mixed speech by multiple speakers. Such instability severely disrupts the contextual coherence required for language recognition. By incorporating this stability indicator into a conflict indicator system, the system can identify such non-steady-state inputs and use them as a suppression factor to prevent language recognition result fluctuations caused by VAD jitter from being misjudged as speech direction misselection conflicts.

[0157] This embodiment introduces an input speech quality dimension index, which works organically with the aforementioned language identification confidence dimension index. Specifically, the language identification confidence dimension reflects the model's subjective probabilistic judgment on language classification, while the input speech quality dimension provides objective environmental constraints on the credibility of this judgment. Through this synergy, the system can not only identify surface conflicts where the confidence of languages ​​outside the set is higher than that of languages ​​within the set, but also further determine whether the conflict is based on reliable speech data. For example, when a candidate language outside the set has extremely high confidence but the input speech duration is extremely short or the signal-to-noise ratio is extremely low, the system uses the quality dimension index to suppress the high-confidence result, classifying it as a low-confidence conflict or no conflict; conversely, when all speech quality indicators are excellent and the candidate language outside the set has a clear advantage, the quality dimension index plays a positive enhancing role, prompting the system to quickly classify it as a high-confidence conflict. This dynamic adjustment mechanism based on quality perception ensures that the speech direction misselection detection can respond promptly when a real misselection occurs, while remaining silent when the data quality is poor, achieving a balance between detection sensitivity and anti-interference capability.

[0158] 5. Indicators based on the number of consecutive conflicts.

[0159] Specifically, the conflict indicator may include: the consecutive rounds of candidate languages ​​outside the first set identified in the current translation round. These consecutive rounds represent the number of times a candidate language outside the first set identified in the current translation round has been consecutively selected as a candidate language outside the first set in this round and prior to this round.

[0160] This embodiment introduces a temporal continuity dimension, upgrading conflict determination from a single translation round to time series analysis. The consecutive rounds are dynamically accumulated count variables, whose update mechanism depends on the consistency comparison between the first out-of-set candidate language determined in the current round and the first out-of-set candidate language recorded in the previous round. Specifically, the system can maintain a session state memory to record the identifier of the first out-of-set candidate language in the previous round and its corresponding consecutive count value. When entering a new translation round, steps S100-S110 are first executed to obtain the first out-of-set candidate language for this round. Subsequently, the first candidate language outside the set in this round is compared with the first candidate language outside the set recorded in the previous round: if the two language identifiers are the same, and the confidence level of the first candidate language outside the set in this round exceeds the preset effective threshold (e.g., 0.6), then the continuous count value of the previous round is incremented by 1, which is used as the continuous round in this round; if the two language identifiers are different, or the confidence level of the first candidate language outside the set in this round is lower than the effective threshold (indicating that it may be due to noise or random fluctuations caused by mixed speech), then the continuous round is reset to 1 or 0 (depending on the specific implementation strategy, usually resetting to 1 indicates the start of a new round).

[0161] For example, suppose a user communicates in Russian throughout a series of multi-turn conversations, while the device remains in a Chinese-English configuration. In the first turn, Russian is detected as a candidate language outside the first set, and the consecutive turn count is 1. In the second turn, Russian is again detected as a candidate language outside the first set, and the consecutive turn count is increased to 2. In the third turn, Russian is still detected, and the consecutive turn count is increased to 3. In this case, the consecutive turn count is 3. Conversely, if the first turn is Russian, the second turn is misidentified as French due to environmental noise, and the third turn reverts to Russian, the consecutive turn count for the third turn will be reset to 1 because a language jump occurred, disrupting the continuity. Through this mechanism, the system can effectively filter out occasional conflict signals caused by short noises, single-sentence foreign language insertions, or recognition jitter. Only when the same candidate language outside the set appears consistently and stably over time is a higher consecutive turn count weight assigned. This consecutive turn count directly reflects the persistence and conversational characteristics of the user's language misselection, and is key evidence for distinguishing between accidental interference and genuine language misselection.

[0162] This embodiment introduces a consecutive-round conflict index, enabling the system to construct a more robust conflict determination logic. A single high-confidence difference may not be sufficient to trigger a strong error correction strategy, as short speech or background noise can lead to misjudgments; however, when a high-confidence difference coexists with a high number of consecutive rounds, it constitutes strong conversational-level evidence of incorrect selection. This synergy allows the system to accurately distinguish between instantaneous fluctuations and genuine speech direction switching, thereby enabling targeted execution of conversational-level speech direction recommendation or reminder suppression mechanisms in subsequent processing. For example, for conflicts with low consecutive rounds, the system can simply record in the background or provide a weak prompt to avoid disturbing the user; while for conflicts with high consecutive rounds, the system can initiate dual-result output, block erroneous translations, or directly recommend switching to the foreign language of the consecutively occurring set, significantly improving the adaptability of the translation device in complex interactive scenarios and the user experience.

[0163] 6. By combining the conflict indicators from multiple dimensions mentioned above, a comprehensive conflict score can be calculated.

[0164] Specifically, the outputs of the restricted recognition process and the open language verification process are obtained. The outputs of the restricted recognition process include: the restricted recognition language, the first ASR text obtained after recognizing the input speech in the restricted recognition language and its recognition confidence level, and the first translated text and its quality score after translating the first ASR text.

[0165] The output of the open language verification process includes: open candidate languages ​​and their corresponding language recognition confidence scores, the second ASR text obtained after recognizing the input speech using the open candidate language with the highest confidence score and its recognition confidence score, and the second translated text and its quality score after translating the second ASR text.

[0166] Based on this, the process of calculating the conflict index according to the output of the restricted identification process and the output of the open language verification process may include:

[0167] S1. Based on the language identification confidence information of the open candidate languages ​​output by the open language verification process, determine the first candidate language with the highest confidence in the set and its confidence P_open(L_in_best) among the candidate languages ​​in the set, and determine the first candidate language outside the set with the highest confidence and its confidence P_open(L_out_best) among the candidate languages ​​outside the set.

[0168] S2. Calculate the difference D and / or ratio Q between the confidence scores of candidate languages ​​within the first set and the confidence scores of candidate languages ​​outside the first set.

[0169] S3. Calculate the quality difference T between the quality score of the second translated text and the quality score of the first translated text.

[0170] S4. Obtain the consecutive rounds C_history of the candidate languages ​​outside the first set determined in the current translation round. The consecutive rounds represent the number of times the candidate languages ​​outside the first set determined in the current translation round have been consecutively used as candidate languages ​​outside the first set in this round and before this round.

[0171] S5. Calculate the quality score Q_voice of the input speech.

[0172] S6. The confidence scores, differences and / or ratios, quality score differences, consecutive rounds, and quality scores of the input speech of the candidate languages ​​outside the first set obtained in the above steps are weighted and fused to obtain a comprehensive conflict score, which is used as a conflict index.

[0173] Weighted fusion refers to mapping multiple heterogeneous dimensional indicators obtained in the above steps to the same numerical space and combining them linearly or nonlinearly according to preset weight coefficients to generate a unified comprehensive conflict score. The weights of each indicator can be dynamically configured according to specific application scenarios, language similarity, or device performance. For example, for languages ​​with significant pronunciation differences, the weight of language confidence difference can be appropriately increased; for scenarios such as business meetings where translation accuracy is extremely important, the weight of translation quality difference can be increased; and for long dialogue scenarios, the weight of consecutive turns can be increased.

[0174] For example, the comprehensive conflict score can be expressed as:

[0175] S=w1×P_open(L_out_best)+w2×D+w3×log(Q)+w4×T+w5×C_history+w6×Q_voice.

[0176] Here, w1-w6 represent the weight values ​​of different indicators. The meanings of P_open(L_out_best), D, Q, T, C_history, and Q_voice are explained above. Q_voice can be a negative number, that is, when there is a lot of noise in the input speech, a value less than zero can be assigned to the input speech quality to suppress the overall conflict score.

[0177] The overall conflict score, as a scalar value, directly quantifies the overall degree of conflict between the current input speech and the restricted speech direction configuration. The higher the score, the greater the likelihood of incorrect speech direction selection and the more substantial the evidence.

[0178] This application integrates multi-source heterogeneous information by constructing a comprehensive conflict score. The confidence scores of candidate languages ​​outside the first set provide basic evidence of language deviation, and the difference and ratio further quantify the significance of this deviation. The quality difference of the translated text verifies the error of restricted speech direction from a semantic level, making up for the shortcomings of simple acoustic recognition. The cumulative effect of the time dimension in consecutive rounds eliminates instantaneous interference and enhances the robustness of the judgment. The input speech quality score serves as a confidence adjustment factor, ensuring a conservative strategy under low-quality signals. These technical features work together through a weighted fusion mechanism to generate the comprehensive conflict score, enabling the final conflict index to not only keenly capture significant speech direction misselection but also effectively suppress false alarms caused by noise, short speech, or similar languages, thus laying a solid foundation for subsequent accurate classification and intelligent error correction.

[0179] In some embodiments of this application, an optional implementation process is described to determine whether there is a speech direction misselection conflict in the input speech based on a conflict index.

[0180] In this embodiment, the speech direction misselection conflict level corresponding to the input speech can be determined based on the conflict index according to the configured conflict level classification rules.

[0181] The conflict level classification rule is a pre-defined logical judgment criterion used to map conflict indicators to discrete semantic conflict levels. This rule can be implemented using different combinations of conflict thresholds depending on the specific composition of the conflict indicators.

[0182] Furthermore, this application can set different conflict thresholds for different language pairs within the restricted language set. For example, a lower conflict threshold can be used for language pairs with significant pronunciation differences; a higher conflict threshold or requiring multiple rounds of confirmation can be used for similar languages ​​or common mixed-language scenarios.

[0183] Among them, conflict indicators can take many forms, including but not limited to the 1-5 different dimensions of conflict indicators described in the aforementioned embodiments, as well as the comprehensive conflict score.

[0184] In some possible implementations, the level of misdirection conflict can be determined based on several different conflict indicators from 1 to 5. In other possible implementations, the level of misdirection conflict can be determined directly based on the comprehensive conflict score.

[0185] The speech direction misselection conflict level includes at least three levels. This multi-level classification mechanism can avoid the false alarm or false alarm problems caused by traditional binary judgment (i.e., only setting two states: conflict and no conflict). By introducing an intermediate level, the system can distinguish between two scenarios: insufficient evidence but suspected risk and high probability misselection with conclusive evidence.

[0186] For example, if the conflict index is a comprehensive conflict score S (ranging from 0 to 1), the conflict level classification rule can be configured as follows: when S < 0.3, it is determined to be the first level (indicating that there is no misselection conflict in the linguistic direction); when 0.3 ≤ S < 0.7, it is determined to be the second level (indicating a low-confidence or medium-confidence misselection conflict in the linguistic direction, meaning that the candidate languages ​​outside the set have a certain advantage but it is not significant, or the confidence level fluctuates due to interference from factors such as noise and short speech); when S ≥ 0.7, it is determined to be the third level (indicating a high-confidence misselection conflict in the linguistic direction, meaning that the candidate languages ​​outside the set have an overwhelming advantage over the candidate languages ​​within the restricted set, and the difference in translation quality is significant).

[0187] This step transforms quantified conflict indicators into qualitative conflict levels, providing clear triggering conditions for subsequent implementation of differentiated error correction strategies. The establishment of different levels directly determines whether the system remains silent, logs, issues weak prompts, or performs blocking error correction, thereby achieving refined management of the risk of incorrect speech direction selection.

[0188] This application discretizes continuous conflict indicators into at least three levels of misdirection conflict based on configured conflict level classification rules, achieving a shift from coarse-grained judgment to refined risk control. The level representing the absence of misdirection conflict ensures a zero-interference experience in normal translation scenarios, allowing the device to output results smoothly like in traditional modes. The remaining levels, representing different degrees of misdirection conflict, construct a gradient system for risk response. Using this hierarchical mechanism, the system can dynamically match corresponding processing strategies based on the conflict intensity. For example, it silently executes a limited recognition process when there is no conflict; it only records data in the background to accumulate contextual information when there is a low-confidence conflict; it uses non-blocking weak prompts to guide user attention when there is a medium-confidence conflict; and it activates a strong intervention mechanism when there is a high-confidence conflict, such as recommending switching of direction, comparing two results, or even blocking erroneous translations. This ensures that the translation device has an adaptive intelligent response capability when facing the risk of misdirection.

[0189] In some embodiments of this application, taking the conflict levels including no conflict, low confidence conflict, medium confidence conflict, and high confidence conflict as examples, an optional implementation method for determining the speech direction misselection conflict level corresponding to the input speech based on conflict indicators is introduced.

[0190] 1. No conflict

[0191] When the confidence level P_open(L_out_best) of the candidate language outside the first set is lower than the minimum effective confidence threshold, or when P_open(L_out_best) is not higher than the confidence level P_open(L_in_best) of the candidate language inside the first set and D, Q, and T do not reach the preset threshold, the speech direction misselection conflict level corresponding to the input speech can be determined to be conflict-free.

[0192] In another optional implementation, if the overall conflict score S does not reach the preset threshold, the speech direction misselection conflict level corresponding to the input speech can be determined to be conflict-free.

[0193] 2. Low-confidence conflict

[0194] When the confidence level P_open(L_out_best) of the candidate language outside the first set is greater than P_open(L_in_best), but the confidence difference does not exceed the first set advantage gap threshold, it indicates that the advantage of the candidate language outside the first set relative to the candidate language inside the first set is not obvious. At this time, it can be determined that the speech direction misselection conflict level corresponding to the input speech is a low confidence conflict.

[0195] In another alternative implementation, a low-confidence conflict can also be identified when D, Q, and T only reach the low-confidence threshold.

[0196] In another optional implementation, if the overall conflict score S falls within a preset low confidence score range, the speech direction misselection conflict level corresponding to the input speech can also be determined as a low confidence conflict.

[0197] 3. Conflict of confidence

[0198] When the confidence level P_open(L_out_best) of the candidate language outside the first set is greater than P_open(L_in_best), and the confidence level difference exceeds the first set advantage gap threshold but does not exceed the second set advantage gap threshold, it indicates that the candidate language outside the first set has a certain advantage over the candidate language inside the first set, but it is not enough to directly replace the result of the restricted recognition process. At this time, it can be determined that it belongs to the medium confidence conflict.

[0199] In another alternative implementation, a low-confidence conflict can also be identified when Q and T reach the medium confidence threshold.

[0200] In another optional implementation, if the overall conflict score S falls within the preset medium confidence score range, the speech direction misselection conflict level corresponding to the input speech can also be determined as a medium confidence conflict.

[0201] 4. High-confidence conflict

[0202] When the confidence level P_open(L_out_best) of the candidate language outside the first set is greater than P_open(L_in_best), and the confidence level difference exceeds the second set advantage gap threshold, it indicates that the candidate language outside the first set has a significant advantage over the candidate language inside the first set, and it can be determined to be a high-confidence conflict.

[0203] In another optional implementation, when the device is currently in a restricted translation mode with key binding language, if the candidate language outside the first set is different from the source language currently bound to the key, and the confidence level P_open(L_out_best) of the candidate language outside the first set is greater than the set high confidence threshold, it can be determined that it belongs to a high confidence conflict.

[0204] In another alternative implementation, a high-confidence conflict can also be identified when Q and T reach the high-confidence threshold.

[0205] In another optional implementation, if the overall conflict score S falls within the preset high confidence score range, the speech direction misselection conflict level corresponding to the input speech can also be determined as a high confidence conflict.

[0206] In another optional implementation, when the number of consecutive rounds C_history for candidate languages ​​outside the first set exceeds a preset threshold, and the confidence level of candidate languages ​​outside the first set exceeds the minimum confidence threshold in each round, the speech direction misselection conflict level corresponding to the input speech can also be determined as a high-confidence conflict.

[0207] In another optional implementation, when the recognition confidence of the first ASR text is lower than a set first ASR recognition confidence threshold, and the recognition confidence of the second ASR text is higher than a set second ASR recognition confidence threshold, the speech direction misselection conflict level corresponding to the input speech can also be determined to be a high-confidence conflict. Here, the second ASR recognition confidence threshold is greater than the first ASR recognition confidence threshold.

[0208] The above embodiments only illustrate some optional implementation strategies for determining conflict levels based on different types of conflict indicators. In addition, combinations of one or more other types of conflict indicators can be used to determine the conflict level according to the set conflict level classification rules. This application will not exhaustively list them.

[0209] In some embodiments of this application, after determining the speech direction misselection conflict level corresponding to the input speech, the translation result and / or prompt information can be output using an output strategy that matches the speech direction misselection conflict level.

[0210] The level of discourse misselection conflict can usually be divided into several levels, such as no conflict, low confidence conflict, medium confidence conflict and high confidence conflict, to characterize the credibility of discourse misselection risk.

[0211] Output strategy can refer to the specific combination of interactive actions performed by the translation device according to the determined conflict level. Its content includes the presentation form of the translation result (such as outputting only the restricted process result, outputting both process results at the same time, blocking the restricted process result, etc.) and the triggering method of prompt information (such as no prompt, background recording, non-blocking weak prompt, strong prompt pop-up, etc.).

[0212] In this embodiment, a matching relationship between conflict level and output strategy can be established in advance to ensure the smoothness of the translation process in low-risk scenarios, while providing timely error correction guidance in high-risk scenarios.

[0213] This application's embodiments drive differentiated output of translation results and prompts based on the conflict level of the misselection of language direction, achieving intelligent adaptive human-computer interaction. Specifically, by utilizing the constraint effect of the conflict level on the output strategy, the device can take appropriate countermeasures when faced with language direction conflicts of different degrees of certainty. For example, it can exercise restraint to maintain user experience when evidence is insufficient, and decisively intervene to correct translation errors when evidence is sufficient.

[0214] Furthermore, embodiments of this application illustrate examples of the correspondence between different speech-direction misselection conflict levels and output strategies, referencing... Figure 5 As shown:

[0215] 1. When there is no misselection conflict in the representation of the misselection conflict level, the translation result can be output according to the restricted recognition process.

[0216] The absence of language selection conflict can mean that, based on the aforementioned conflict indicators (such as the probability difference between inside and outside the set, probability ratio, translation quality difference, etc.), there is no significant contradiction between the current input speech and the restricted language set configured by the user. In this state, the system determines that the current restricted recognition process (i.e., the user-preset bilingual automatic judgment or key-bound language, etc.) can accurately reflect the true language of the input speech.

[0217] When the conflict level is determined to be non-conflictical, the translation device directly calls the result generated by the restricted recognition process for output, such as outputting the first recognized text and the first translated text obtained by the restricted recognition process. For example, in the user-configured automatic Chinese-English bilingual judgment mode, if the input voice is confirmed to be English, and the open language verification process shows that the probability of foreign languages ​​such as Russian and Japanese is extremely low, the system will directly broadcast or display the Chinese translation obtained by recognizing and translating from English.

[0218] This processing strategy ensures a smooth and low-latency translation process while maintaining correct context configuration, avoiding unnecessary system resource consumption and user interaction interference, and ensuring an unaffected translation experience.

[0219] 2. When the conflict level of misselection of speech direction represents a low confidence conflict level, the translation result is output according to the restricted identification process, and the conflict index calculated in the current translation round is recorded in the background.

[0220] Low-confidence conflict levels typically correspond to scenarios where there are potential signs of inconsistency in discourse, but the strength of the evidence is insufficient. In such cases, to avoid accidental interruption, the translation results can continue to be output according to the restricted recognition process, and the conflict index calculated in the current translation round can be recorded in the background without outputting any prompts.

[0221] The record for this conflict indicator includes data such as the first set of candidate languages ​​outside the current round and their confidence levels, confidence difference D, confidence ratio Q, ASR identification confidence difference, and translation quality score difference T. This data is written to the session state memory for subsequent rounds of conflict assessment.

[0222] For example, when a user inputs the extreme phrase "OK," the open language verification process might slightly favor French, but the restricted recognition process would still classify it as English; in this case, the difference is not significant. The system outputs the English translation as usual, while simultaneously recording the low-probability French feature. If a similar French tendency reappears in the next round of speech, the accumulated conflict indicators will help improve the confidence of subsequent judgments.

[0223] By recording silently in the background instead of immediately interrupting the user, this step effectively avoids false alerts caused by short voice messages, noise, or fluctuations in similar languages. While ensuring the continuity of translation, it also accumulates a key data foundation for multi-round comprehensive judgment.

[0224] 3. When the confidence conflict level is in the speech direction misselection conflict level representation, the translation result is output according to the restricted recognition process, and a non-blocking prompt strategy is implemented. The non-blocking prompt strategy is to output speech direction misselection prompts to the user while maintaining the normal output of translation results in the restricted recognition process, and the prompts do not block the current translation process.

[0225] The medium confidence conflict level indicates that candidate languages ​​outside the first set have shown a certain probabilistic or qualitative advantage over candidate languages ​​within the first set, but have not yet reached the point where a forced switch or block is necessary. At this point, the system suspects that the language configuration may be incorrect, but still retains a certain degree of uncertainty.

[0226] The specific implementation methods of the non-blocking prompt strategy include, but are not limited to: displaying weak prompt text in the status bar or corner area of ​​the translation device's screen, such as "The current speech may not be the selected language," "A suspected language has been detected; would you like to switch to that language to the target language?", popping up a semi-transparent candidate language switching entry card, or playing a soft prompt sound. These prompt actions occur in parallel with the broadcast or display of the main translation result and will not pause, overwrite, or delay the translation result generated by the restricted recognition process.

[0227] This strategy informs users of potential risks while respecting their current operating habits and the continuity of translation. Users can choose to ignore the prompts and continue communicating, or manually switch to the corrective mode when convenient, achieving a balance between error correction prompts and user experience.

[0228] 4. When the speech direction misselection conflict level represents a high-confidence conflict level, it means there is sufficient evidence to suggest that the current input speech does not belong to the restricted language set, but rather to a language outside that specific set. In this case, the system can perform any one or more of the following differential processing steps:

[0229] 4.1 Prompt the user to switch the source language to the recommended language, which is the open candidate language with the highest confidence level.

[0230] 4.2 Simultaneously output the translation results of the restricted recognition process and the translation results of the open language verification process.

[0231] Specifically, the system can output two results in parallel, simultaneously displaying or broadcasting two sets of translation results. One set is based on the user's original configuration (restricted recognition process), and the other set is based on the recommended language (open language verification process). For example, the screen may display the original configuration translation (which may be inaccurate) and the recommended language translation (high confidence) in a split-screen format, allowing the user to compare and judge for themselves.

[0232] 4.3. Blocking the translation results output by the restricted recognition process.

[0233] Specifically, the system temporarily suppresses or blocks the translation output of the restricted recognition process, prioritizing user confirmation of the speech direction to prevent erroneous information from misleading the user. This is especially important in high-risk scenarios such as healthcare and law.

[0234] 4.4 Prompt for incorrect use of the voice selection button.

[0235] In key binding language mode, for example, if a user presses the English to Chinese key, but the open language verification process strongly indicates that the input is Russian, the system outputs a prompt: The current key binding is English, but the input is more likely to be Russian. Do you want to change it to Russian to Chinese?

[0236] This application employs the aforementioned tiered output strategy. The restricted recognition process ensures the continuity of basic translation functions, while the open language verification process provides a basis for error correction. The combination of non-blocking prompts and high-confidence, strong intervention avoids both low-frequency false alarms from disturbing users and ensures that high-frequency, definitive errors are corrected promptly. This mechanism enables the translation device to dynamically perceive and correct incorrect language selection without altering existing user operating habits (such as button binding or bilingual mode), thereby significantly improving the accuracy and efficiency of cross-language communication.

[0237] In some optional embodiments, the translation device receives user feedback on the prompts and writes the feedback into the current session state. If the user confirms switching to the recommended language, the device can update the other party's language to the recommended language within the current session, for example, from English to Russian. If the user ignores the prompts or explicitly refuses to switch, the device can reduce the frequency of the same prompts within the same session to avoid repeated interruptions.

[0238] In some alternative implementations, the device does not directly and permanently override the global language setting, but only applies to the current round or the current session. For example, if the user confirms "Russian to Chinese" in the current conversation, then Russian to Chinese will be used first in the current session; after the session ends, the device can still retain the user's original Chinese-English configuration, or ask whether to continue using Russian through a prompt on the next startup.

[0239] In some optional implementations, if the same candidate language outside the first set triggers high-confidence conflicts in multiple consecutive rounds, and the user confirms the recommended language direction multiple times (reaching a set threshold), the device can increase the recommendation priority of that language direction in subsequent conversations. For example, if the user confirms Russian to Chinese three times consecutively, the device can recommend Russian earlier when it detects a similar scenario next time.

[0240] The following embodiments further provide some optional examples of the method of this application:

[0241] Example 1: The user selects the automatic bilingual (Chinese and English) judgment mode, but the other party is actually speaking Russian. The device's limited recognition process interprets the input speech as English, resulting in low-quality English text and inaccurate Chinese translations. The open language verification process outputs: Russian: 0.91 confidence level, English: 0.04 confidence level, Chinese: 0.02 confidence level. The device calculates the conflict index and determines that Russian, which is outside the limited set, has a higher confidence advantage over English, which is within the set. Therefore, it prompts "The other party may be speaking Russian. Would you like to switch from Russian to Chinese?" and simultaneously displays candidate Russian-to-Chinese translations.

[0242] Example 2: A user presses the "English to Chinese" physical button, but the actual speaker's voice is Russian. The device obtains a low-confidence result by following the English to Chinese path, while simultaneously using the open language verification process to determine that Russian has a high confidence level. The device displays the message "Current button is bound to English to Chinese, but the detected input is more likely to be Russian," and provides an option to "Translate from Russian to Chinese this time."

[0243] Example 3: When the user inputs a very short voice "OK", the open language verification process identifies French with the highest confidence level, while the restricted recognition process still identifies it as English. In this case, the confidence level distribution output by the open language verification process shows no significant difference between French and English confidence levels. The device classifies this as a low-confidence conflict level and outputs the English translation as usual, without providing a language switching prompt to avoid interfering with normal use. Simultaneously, the low-confidence feature of French can be recorded. If a similar French tendency reappears in the next round of voice input, the accumulated conflict indicators will help improve the confidence level of subsequent judgments.

[0244] In some embodiments of this application, a speech direction misselection detection device is provided, which may include:

[0245] The module includes a voice acquisition module, a user speech configuration module, a restricted recognition and translation module, an open language verification module, and a conflict judgment module, among which:

[0246] The voice acquisition module is used to acquire input voice.

[0247] The user language configuration module is used to obtain the translation mode and language configuration currently selected by the user. This configuration includes, but is not limited to: fixing the user's language, the target language, whether to enable bilingual automatic judgment, whether to enable multilingual automatic judgment, the binding relationship between physical or virtual buttons and language, the currently triggered button, and the language configuration of the current conversation history.

[0248] The restricted recognition translation module is used to perform the restricted recognition process when the device is currently in restricted translation mode. That is, it performs language recognition on the input speech according to the restricted language set. For example, in the Chinese-English bilingual automatic judgment mode, it only calculates or selects the source language between Chinese and English; in the key binding mode, it determines the source language and target language according to the key that is pressed.

[0249] The open language verification module is used to perform an open language verification process on the input speech, that is, to perform language recognition on the same input speech without being constrained by the restricted language set, so as to obtain the language recognition confidence information of the open candidate languages. The open candidate languages ​​include candidate languages ​​within the set that belong to the restricted language set, and candidate languages ​​outside the set that do not belong to the restricted language set.

[0250] The conflict determination module is used to determine whether there is a speech direction misselection conflict in the input speech based on the output of the restricted recognition process and the output of the open language verification process.

[0251] Further optionally, the aforementioned speech direction misselection conflict may include multiple different conflict levels, then the speech direction misselection detection device may further include:

[0252] The interactive output module is used to output translation results and / or prompts based on the level of speech misselection conflict corresponding to the input speech, and adopts an output strategy that matches the level of speech misselection conflict.

[0253] The detailed data processing procedures for each of the above modules can be found in the detailed description of the methods section above, and will not be repeated here.

[0254] This application also provides a translation device in its embodiments. (See reference...) Figure 6 The diagram illustrates a structure suitable for implementing the translation device in the embodiments of this application. The translation device in the embodiments of this application may include, but is not limited to, fixed terminals such as translators, voice recorders, mobile phones, tablets, large-screen teaching displays, wearable devices, etc. Figure 6 The translation device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0255] like Figure 6 As shown, the translation device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 2 or a program loaded from a storage device 8 into a random access memory (RAM) 3, to implement the speech direction misselection detection method of the foregoing embodiments of this application. When the translation device is powered on, the RAM 3 also stores various programs and data required for the operation of the translation device. The processing unit 1, ROM 2, and RAM 3 are interconnected via a bus 4. An input / output (I / O) interface 5 is also connected to the bus 4.

[0256] Typically, the following devices can be connected to I / O interface 5: input devices 6 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 7 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 8 including, for example, memory cards, hard drives, etc.; and communication devices 9. Communication device 9 allows the translation device to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Translation devices with various means are shown, but it should be understood that it is not required to implement or have all of the means shown. More or fewer means may be implemented or included alternatively.

[0257] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement the speech direction misselection detection method of any of the translation devices provided in this application.

[0258] This application also provides a computer-readable storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the speech direction misselection detection method of any of the translation devices provided in this application.

[0259] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0260] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0261] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0262] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0263] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

Claims

1. A method for detecting misselected speech direction in a translation device, characterized in that, include: When the translation device is currently in restricted translation mode, the input speech is acquired. In the restricted translation mode, the translation device is only allowed to perform language recognition on the input speech within a restricted set of languages. The process involves executing a restricted recognition process and an open language verification process. The restricted recognition process involves recognizing the language of the input speech according to the restricted language set. The open language verification process involves recognizing the language of the input speech without being constrained by the restricted language set, in order to obtain language recognition confidence information for open candidate languages. The open candidate languages ​​include candidate languages ​​within the restricted language set and candidate languages ​​outside the restricted language set. Based on the output of the restricted recognition process and the output of the open language verification process, it is determined whether the input speech has a speech direction misselection conflict.

2. The method according to claim 1, characterized in that, The process of determining whether the input speech has a speech direction misselection conflict based on the output of the restricted recognition process and the output of the open language verification process includes: Based on the output of the restricted recognition process and the output of the open language verification process, a conflict index is calculated. The conflict index is used to measure the degree of conflict between the input speech and the current speech direction configuration. Based on the conflict index, it is determined whether the input speech has a speech direction misselection conflict.

3. The method according to claim 2, characterized in that, The conflict indicators include: indicators of language identification confidence dimension; The process of calculating the conflict index of the speech recognition confidence dimension includes: Based on the language identification confidence information of the open candidate languages ​​output by the open language verification process, the first candidate language with the highest confidence and its confidence are determined among the candidate languages ​​within the set, and the first candidate language outside the set with the highest confidence and its confidence are determined among the candidate languages ​​outside the set. The conflict index of the speech recognition confidence dimension is determined based on the confidence scores of candidate languages ​​within the first set and the confidence scores of candidate languages ​​outside the first set.

4. The method according to claim 3, characterized in that, The conflict indicators for the confidence dimension of speech recognition include: The difference or ratio between the confidence scores of candidate languages ​​within the first set and the confidence scores of candidate languages ​​outside the first set.

5. The method according to claim 3, characterized in that, The output of the restricted recognition process includes: the restricted recognition language, the first ASR text obtained after recognizing the input speech according to the restricted recognition language, and the recognition confidence level; The output of the open language verification process includes: open candidate languages ​​and their corresponding language recognition confidence scores, and the second ASR text obtained after recognizing the input speech using the open candidate language with the highest confidence score, along with its recognition confidence score. The conflict index also includes an index for ASR recognition confidence dimension, which includes: recognition confidence of the first ASR text, recognition confidence of the second ASR text, and / or, the difference in recognition confidence between the first ASR text and the second ASR text.

6. The method according to claim 3, characterized in that, The output of the restricted recognition process includes: the restricted recognition language, the first ASR text obtained after recognizing the input speech according to the restricted recognition language, the first translated text after translating the first ASR text and its quality score; The output of the open language verification process includes: open candidate languages ​​and their corresponding language recognition confidence scores, the second ASR text obtained after recognizing the input speech using the open candidate language with the highest confidence score, and the second translated text and its quality score after translating the second ASR text. The conflict index also includes indicators for the quality dimension of the translated text, which include: the quality score of the first translated text, the quality score of the second translated text, and / or the difference in quality scores between the first and second translated texts.

7. The method according to claim 3, characterized in that, The conflict indicators also include: The metrics for input speech quality include at least one of the following: input speech duration, input speech signal-to-noise ratio, and metrics characterizing the stability of input speech VAD detection results.

8. The method according to claim 3, characterized in that, The conflict indicators also include: The consecutive rounds of the candidate languages ​​outside the first set determined in the current translation round, wherein the consecutive rounds represent the number of times the candidate languages ​​outside the first set determined in the current translation round have been consecutively used as candidate languages ​​outside the first set in the current round and before the current round.

9. The method according to claim 2, characterized in that, The output of the restricted recognition process includes: the restricted recognition language, the first ASR text obtained after recognizing the input speech according to the restricted recognition language and the recognition confidence, and the first translated text after translating the first ASR text and its quality score. The output of the open language verification process includes: open candidate languages ​​and their corresponding language recognition confidence scores, the second ASR text obtained after recognizing the input speech using the open candidate language with the highest confidence score and its recognition confidence score, and the second translated text and its quality score after translating the second ASR text. The process of calculating the conflict index based on the output of the restricted identification process and the output of the open language verification process includes: Based on the language identification confidence information of the open candidate languages ​​output by the open language verification process, the first candidate language with the highest confidence and its confidence are determined among the candidate languages ​​within the set, and the first candidate language outside the set with the highest confidence and its confidence are determined among the candidate languages ​​outside the set. Calculate the difference and / or ratio between the confidence scores of candidate languages ​​within the first set and the confidence scores of candidate languages ​​outside the first set; Calculate the quality difference between the second translated text quality score and the first translated text quality score; Obtain the consecutive rounds of the first set of candidate languages ​​determined in the current translation round, wherein the consecutive rounds represent the number of times the first set of candidate languages ​​determined in the current translation round has been consecutively used as the first set of candidate languages ​​in the current round and before the current round; Calculate the quality score of the input speech; The confidence scores, differences and / or ratios, quality score differences, consecutive rounds, and quality scores of the input speech of candidate languages ​​outside the first set are weighted and fused to obtain a comprehensive conflict score, which is used as the conflict index.

10. The method according to claim 2, characterized in that, The process of determining whether the input speech has a speech direction misselection conflict based on the conflict index includes: According to the configured conflict level classification rules, the speech direction misselection conflict level corresponding to the input speech is determined based on the conflict index. The speech direction misselection conflict level includes at least three levels, one of which represents that there is no speech direction misselection conflict, and the remaining levels represent different degrees of speech direction misselection conflict.

11. The method according to claim 10, characterized in that, Also includes: Based on the speech direction misselection conflict level corresponding to the input speech, the translation result and / or prompt information are output using an output strategy that matches the speech direction misselection conflict level.

12. The method according to claim 11, characterized in that, The process of outputting translation results and / or prompts using an output strategy that matches the level of speech misselection conflict includes: When there is no misdirection conflict in the described linguistic misselection conflict level representation, the translation result is output according to the restricted identification process. When the misselection conflict level represents a low-confidence conflict level, the translation result is output according to the restricted identification process, and the conflict index calculated in the current translation round is recorded in the background. When the conflict level is believed in the speech direction misselection conflict level representation, the translation result is output according to the restricted identification process, and a non-blocking prompt strategy is executed. The non-blocking prompt strategy is to output speech direction misselection prompts to the user while maintaining the normal output of the translation result of the restricted identification process, and the prompts do not block the current translation process. When the misselection conflict level represents a high-confidence conflict level, perform one or more of the following processes: The user is prompted to switch their source language to the recommended language, which is the open candidate language with the highest confidence level. Simultaneously output the translation results of the restricted recognition process and the translation results of the open language verification process; Block the translation results output by the restricted recognition process; Incorrect use of the voice selection button.

13. The method according to any one of claims 1-12, characterized in that, The open candidate languages ​​include all languages ​​supported by the translation device; Alternatively, the open candidate languages ​​are a set of language seeds selected from all languages ​​supported by the translation device according to preset filtering rules, and the open candidate languages ​​include at least one candidate language that does not belong to the restricted language set.

14. A translation device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement each step of the speech direction misselection detection method for the translation device as described in any one of claims 1 to 13.

Images