A keyword recognition based multi-modal grouping translation method and system
By using a keyword-based multimodal grouping translation method, the problem of frequent resource switching in machine translation systems when processing multiple different types of sequences to be translated is solved, achieving more efficient resource utilization and improved user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- IOL WUHAN INFORMATION TECH CO LTD
- Filing Date
- 2022-10-11
- Publication Date
- 2026-07-07
AI Technical Summary
Existing machine translation systems, when processing various types of sequences to be translated, frequently switch processes and call resources, leading to increased system overhead, affecting real-time output quality, and reducing user experience.
A multimodal grouping translation method based on keyword recognition is adopted. By identifying multimodal keywords and generation time in the sequence to be translated, the sequence is grouped and the corresponding translation containers are scheduled for translation, ensuring resource and process matching and avoiding frequent switching.
This reduces system resource fragmentation, improves the real-time output of translation results and user experience, and ensures the synchronization and accuracy of translation results.
Smart Images
Figure CN115510878B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of translation technology, and in particular relates to a multimodal group translation method and system based on keyword recognition. Background Technology
[0002] Traditional machine translation can already convert a corpus to be translated from a source language sequence to a target language sequence and output it. For example, it can translate the text / speech to be translated in language A into the translation in language B and output it in text or speech format, achieving real-time or near real-time translation results.
[0003] With the widespread adoption of multimedia smart terminal devices, the output effects of machine translation are becoming increasingly diverse. For certain sequences to be translated containing special content, rich media translation terminals can display the translation effects in various multimedia formats, such as video, animation, and audio overlay. Correspondingly, more system resources are needed to support multimedia format output.
[0004] In existing technologies, a corresponding translation model needs to be established for each multimedia output format to activate the corresponding resource process. For example, when outputting video, a video process needs to be established, including calling the CPU process and the graphics card process; when outputting plain text, it is necessary to switch to a separate CPU process to call the CPU core; and when outputting audio, it is necessary to switch to the audio process to call the sound card resources.
[0005] When multiple source terminals simultaneously generate various types of sequences to be translated, and these sequences have different output styles, if the sequences are input one by one into the physical translator using traditional methods, the physical translator needs to frequently switch between different processes to access different types of physical resources. Frequent process switching and frequent access to system resources increase system overhead and fragmentation, leading to system lag, affecting the real-time output of translation results, and degrading the user experience. Summary of the Invention
[0006] To address some or all of the aforementioned technical problems, this invention proposes a multimodal grouping translation method and system based on keyword recognition.
[0007] Specifically, in a first aspect of the present invention, a multimodal grouping translation method based on keyword recognition is proposed, the method comprising the following steps:
[0008] The sequence identification step is used to receive the sequence to be translated and identify the multimodal keywords and sequence generation time.
[0009] Grouping steps for sequences to be translated: Based on the multimodal keywords and sequence generation time, add each sequence to be translated to the corresponding grouping queue;
[0010] Group translation steps: For each group, schedule the corresponding translation container to perform the translation;
[0011] The multimodal keywords are used to characterize the translation modality of the sequence to be translated, and the translation modality includes one or any combination of plain text translation, plain speech translation, image-text translation, and video translation.
[0012] The difference in generation time between any two sequences to be translated contained in each group queue is less than a preset grouping time threshold, and the preset grouping time thresholds are different for different group queues.
[0013] As a further improvement, the preset grouping time threshold corresponding to each group queue is determined by the number of sequences to be translated contained in the group queue, the number of translation modes of the sequences to be translated contained in the group queue, and the sequence generation time of the sequences to be translated contained in the group queue.
[0014] Specifically, the method is executed by a group translation terminal, which is configured with multiple translation containers. Each translation container runs a virtual translator. Different virtual translators share the physical resources of the group translation terminal in a time-sharing manner or in parallel and real-time in groups.
[0015] As a specific improvement, the physical resources include one or any combination of GPU resources, video memory resources, sound card resources, and CPU resources.
[0016] Each group queue contains sequences of the same or partially the same translation modality.
[0017] Different translation modalities correspond to different types of virtual translators that can be scheduled with different types of physical resources.
[0018] As a further improvement, the group translation step specifically includes:
[0019] Determine the translation modality of the sequence to be translated contained in each group queue;
[0020] The corresponding translation container is determined based on the translation modality.
[0021] In a second aspect of the invention, a multimodal group translation system based on keyword recognition is provided, the system including a multimodal keyword database for performing the method described in the first aspect.
[0022] Specifically, the multimodal keyword database pre-stores multiple multimodal keywords, which are used to characterize the translation modality of the sequence to be translated. The translation modality includes one or any combination of plain text translation, plain speech translation, image-text translation, and video translation.
[0023] The system includes:
[0024] The sequence identification unit is used to receive the sequence to be translated and identify the multimodal keywords and sequence generation time.
[0025] Sequence to be translated grouping unit: Each sequence to be translated is added to the corresponding grouping queue based on the multimodal keywords and sequence generation time;
[0026] Grouped Translation Unit: For each group, the corresponding translation container is scheduled to perform translation;
[0027] In this context, the difference in generation time between any two sequences to be translated contained in each group queue is less than a preset grouping time threshold, and the translation modes of the sequences to be translated contained in each group queue are the same or partially the same; the preset grouping time thresholds are different for different group queues.
[0028] In terms of hardware architecture, the translation container runs on a group translation terminal. Each translation container runs a virtual translator. Different virtual translators share the physical resources of the group translation terminal in a time-sharing manner or in parallel real-time group sharing of the physical resources of the group translation terminal.
[0029] The physical resources include one or any combination of GPU resources, video memory resources, sound card resources, and CPU resources.
[0030] The group translation unit determines the translation modality of the sequence to be translated contained in each group queue, and determines the translation container corresponding to each group queue based on the translation modality;
[0031] If the translation modal of the sequence to be translated contained in the first group queue includes text-image translation and / or video translation, then the virtual translator running the translation container corresponding to the first group queue can schedule the GPU and graphics card resources of the group translation terminal.
[0032] The technical solution of this invention, through a pre-established multimodal keyword database, first identifies the multimodal keywords and sequence generation time upon receiving a sequence to be translated. Then, based on the multimodal keywords and sequence generation time, each sequence to be translated is added to its corresponding group queue. For each group, a corresponding translation container is scheduled for translation, ensuring that the schedulable resources and processes of the translation container entering the group match the translation modality of the sequence to be translated, avoiding frequent process and resource type switching and reducing system resource fragmentation. Simultaneously, the difference in generation time between any two sequences to be translated within each group queue is less than a preset group time threshold, synchronizing the time the translation sequence enters the group queue with the time the translation result is output, preventing sequence misalignment. Finally, different preset group time thresholds for different group queues ensure that the size and generation time difference of each group queue conform to the actual changes in the current group, further ensuring a more reasonable number of translation containers called for that group queue, preventing data transmission delays or translation waiting, and improving the user experience.
[0033] Further embodiments and improvements of the present invention will be described in conjunction with the accompanying drawings and specific examples. Attached Figure Description
[0034] Figure 1 This is a schematic diagram illustrating the steps of a multimodal group translation method based on keyword recognition according to an embodiment of the present invention;
[0035] Figure 2 It is execution Figure 1 A schematic diagram of the hardware architecture of a multimodal group translation method based on keyword recognition;
[0036] Figures 3-4 This is a schematic diagram illustrating different translation modalities of a multimodal group translation method based on keyword recognition according to an embodiment of the present invention;
[0037] Figure 5 This is a schematic diagram of the principle architecture of a multimodal group translation system based on keyword recognition according to an embodiment of the present invention;
[0038] Figure 6 This is a schematic diagram illustrating the working principle of a multimodal group translation system based on keyword recognition, according to an embodiment of the present invention. Detailed Implementation
[0039] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0040] First, see Figure 1 . Figure 1 This is a schematic diagram illustrating the steps of a multimodal group translation method based on keyword recognition according to an embodiment of the present invention.
[0041] Figure 1 The method comprises three main steps: a sequence identification step, a sequence grouping step, and a grouped translation step, each step being performed as follows:
[0042] The sequence identification step is used to receive the sequence to be translated and identify the multimodal keywords and sequence generation time.
[0043] Grouping steps for sequences to be translated: Based on the multimodal keywords and sequence generation time, add each sequence to be translated to the corresponding grouping queue;
[0044] Group translation steps: For each group, schedule the corresponding translation container to perform the translation.
[0045] Figure 2 This is a schematic diagram of the hardware architecture for implementing the keyword recognition-based multimodal group translation method.
[0046] exist Figure 2 In this method, the method is executed by a group translation terminal, which is configured with multiple translation containers. Each translation container runs a virtual translator. Different virtual translators share the physical resources of the group translation terminal in a time-sharing manner or in parallel real-time group sharing of the physical resources of the group translation terminal.
[0047] Next, combined Figures 3-4 The diagram illustrates different translation modalities of a multimodal group translation method based on keyword recognition, and details preferred embodiments of the method of the present invention.
[0048] The sequence identification step is used to receive the sequence to be translated and identify the multimodal keywords and sequence generation time.
[0049] As an example, the sequence to be translated is a first language sequence, which can be a text sequence or a speech sequence;
[0050] The first language sequence can be input by the first user via voice through a microphone device or a text input device, such as a touch device or a keyboard device;
[0051] Preferably, the generation time is added to the first language sequence when the first language sequence is generated;
[0052] The first language sequence usually contains one or more multimodal keywords, which are used to characterize the translation modality of the sequence to be translated. The translation modality includes one or any combination of plain text translation, plain speech translation, image-text translation, and video translation.
[0053] The sequence identification step can identify multimodal keywords in the sequence to be translated after receiving it, thereby determining the translation modality corresponding to the sequence to be translated.
[0054] Specifically, a multimodal keyword database can be pre-constructed, which stores multiple multimodal keywords in advance.
[0055] by Figure 3 For example, Figure 3 An example of the sequence to be translated is "Next, the chemical formula for the combustion of hydrogen in oxygen is shown" in the first language.
[0056] By using a pre-built multimodal keyword database, the following multimodal keywords can be identified:
[0057] Hydrogen, oxygen, chemical formula, display;
[0058] Therefore, the translation mode of the sequence to be translated is determined to be image-text translation, that is, the translation result should display the chemical formula image corresponding to the chemical formula "2H2O+O2=2H2O" in the second language.
[0059] For example, chemical formula images include molecular structure diagrams of "H2O, O2, H2O".
[0060] by Figure 4 For example, Figure 4 An example of the sequence to be translated is "playing footage of hydrogen burning in oxygen" in a third language.
[0061] By using a pre-built multimodal keyword database, the following multimodal keywords can be identified:
[0062] Hydrogen, oxygen, playback, visuals;
[0063] Therefore, the translation mode of the sequence to be translated is determined to be video translation, that is, the translation result should display the combustion scene corresponding to the chemical formula "2H2O+O2=2H2O" in the fourth language.
[0064] In this step, the sequence identification step also identifies the sequence generation time of the sequence to be translated;
[0065] Next, the process of grouping the sequences to be translated is performed, and each sequence to be translated is added to the corresponding grouping queue based on the multimodal keywords and the sequence generation time.
[0066] Specifically, the difference in generation time between any two sequences to be translated contained in each group queue is less than a preset grouping time threshold, and the preset grouping time thresholds are different for different group queues.
[0067] Based on the above grouping, it can be ensured that each group queue subsequently enters the same container, and the schedulable resource type of the container remains unchanged to the greatest extent possible, so as to avoid switching between resource types or different processes.
[0068] The group translation process involves scheduling the corresponding translation container for each group.
[0069] by Figure 3 For example, Figure 3 The input sequence “Next, show the chemical formula for the combustion of hydrogen in oxygen” should schedule the first translation container, and the virtual machine running on the first translation container can schedule GPU resources and video memory resources.
[0070] by Figure 4 For example, Figure 4 The input sequence "playing a picture of hydrogen burning in oxygen" should schedule a second translation container, on which a virtual machine is running that can schedule GPU resources, video memory resources and sound card resources.
[0071] As a specific example, multiple translation containers run on a grouped translation terminal, which has physical resources, including GPU resources, video memory resources, sound card resources, CPU resources, and other resources.
[0072] The present invention is described below using a single group translation terminal as an example, but it is understood that the method of the present invention can be executed in parallel by multiple group translation terminals.
[0073] Suppose a group translation terminal modeA runs three containers Docker1, Docker2, and Docker3. Each Docker executes a virtual translator, denoted as Vm1, Vm2, and Vm3. Different virtual translators share the physical resources of the group translation terminal in a time-sharing manner or in parallel real-time group sharing of the physical resources of the group translation terminal.
[0074] Specifically, it is assumed that the group translation terminal is configured with a 6-core GPU, an 8-core CPU, 1000M video memory, and 500M sound card resources (5 sound cards).
[0075] An example of different virtual translators sharing the physical resources of the grouped translation terminal in a time-sharing manner could be:
[0076] In the first period, Vm1 can access 6-core GPU, 6-core CPU, and 800MB of video memory resources; at this time, Vm2 and Vm3 obviously cannot access GPU resources; in the second period, Vm1 releases all accessed resources, and Vm2 and Vm3 can then access the corresponding physical resources as needed.
[0077] An example of different virtual translators sharing the physical resources of the grouped translation terminals in parallel and in real time could be:
[0078] Preset:
[0079] Vm1 uses 3-core GPU, 3-core CPU, 800M video memory resources and one sound card resource (100M); Vm2 uses 3-core GPU, 2-core CPU, and 100M video memory resources.
[0080] The Vm3 uses 1 CPU core and 3 sound card resources (300M).
[0081] Clearly, the virtual translator Vm1 can perform video translation, the virtual translator Vm2 can perform text-image translation, and the virtual translator Vm3 can perform audio translation (pure speech translation).
[0082] Therefore, for each group, the group translation step can schedule the corresponding translation container for translation.
[0083] Of course, it can also be seen that the virtual translator VM1 can perform translation modes not only including video translation, but also supports text-image translation and pure voice translation.
[0084] Therefore, when adding sequences to a group, the translation modes of the sequences to be translated contained in each group are the same or partially the same.
[0085] For example, sequences to be translated, whose translation modalities are video translation and text-image translation, can be added to the same group queue.
[0086] In summary, the group translation step specifically includes: determining the translation modality of the sequence to be translated contained in each group queue; determining the corresponding translation container based on the translation modality; and the virtual translator corresponding to the translation container of different translation modalities has different types of schedulable physical resources.
[0087] Therefore, the schedulable resources and processes of the translation containers entering in groups are matched with the translation modality of the sequence to be translated itself, avoiding frequent process switching and resource type switching, and reducing the degree of system resource fragmentation.
[0088] As a further improvement, the difference in generation time between any two sequences to be translated contained in each group queue is less than a preset grouping time threshold, and the preset grouping time thresholds are different for different group queues.
[0089] Specifically, the preset grouping time threshold for each group queue is determined by the number of sequences to be translated contained in the group queue, the number of translation modes of the sequences to be translated contained in the group queue, and the sequence generation time of the sequences to be translated contained in the group queue.
[0090] The above method can be implemented automatically using computer program instructions. Taking the determination of the preset grouping time threshold for each grouping queue as an example, the pseudocode of the corresponding computer program instructions is as follows:
[0091] Input: the number K of the sequences to be translated contained in the group queue, the number M of the translation modes of the sequences to be translated contained in the group queue, and the set of sequence generation times SetT of the sequences to be translated contained in the group queue;
[0092] Output: The preset grouping time threshold GroupThreshold corresponding to the grouped queue satisfies the following conditions:
[0093]
[0094] Where Std is the preset grouping time base value.
[0095] In practice, the preset grouping time threshold for each group queue is a baseline value Std. First, the initial elements of the group queue are determined using the baseline value Std. Then, it is calculated whether the preset grouping time threshold of the group queue conforms to the above formula. If it does not conform, the number of elements in the group queue is adjusted (by adding or deleting elements) until the preset grouping time threshold of the group queue conforms to the above formula.
[0096] exist Figures 1-4 Based on this, see Figure 5 . Figure 5 This is a schematic diagram of the principle architecture of a multimodal group translation system based on keyword recognition, according to an embodiment of the present invention.
[0097] exist Figure 5 The present invention illustrates a multimodal group translation system based on keyword recognition, the system comprising a multimodal keyword database.
[0098] The multimodal keyword database pre-stores multiple multimodal keywords, which are used to characterize the translation modality of the sequence to be translated. The translation modality includes one or any combination of plain text translation, plain speech translation, image-text translation, and video translation.
[0099] The system also includes:
[0100] The sequence identification unit is used to receive the sequence to be translated and identify the multimodal keywords and sequence generation time.
[0101] Sequence to be translated grouping unit: Each sequence to be translated is added to the corresponding grouping queue based on the multimodal keywords and sequence generation time;
[0102] Grouped Translation Unit: For each group, the corresponding translation container is scheduled to perform translation;
[0103] In this case, the difference in generation time between any two sequences to be translated contained in each group queue is less than a preset grouping time threshold.
[0104] Figure 5 It is also shown that the translation container runs on a grouped translation terminal, with each translation container running a virtual translator. Different virtual translators share the physical resources of the grouped translation terminal in a time-sharing manner or in parallel real-time grouped sharing of the physical resources of the grouped translation terminal.
[0105] The physical resources include one or any combination of GPU resources, video memory resources, sound card resources, and CPU resources.
[0106] The group translation unit determines the translation modality of the sequence to be translated contained in each group queue, and determines the translation container corresponding to each group queue based on the translation modality;
[0107] If the translation modal of the sequence to be translated contained in the first group queue includes text-image translation and / or video translation, then the virtual translator running the translation container corresponding to the first group queue can schedule the GPU and graphics card resources of the group translation terminal.
[0108] Each group queue contains sequences with the same or partially the same translation modality; different group queues have different preset grouping time thresholds.
[0109] Figure 6 This is a schematic diagram illustrating the working principle of a multimodal group translation system based on keyword recognition, according to an embodiment of the present invention.
[0110] exist Figure 6 In this system, the group translation terminal contains three translation containers: DoA, DoB, and DoC. Each translation container runs a virtual translator.
[0111] All translation containers can schedule CPU resources, while DoA can schedule CPU, GPU, graphics card, and sound card resources; DoC can schedule CPU and graphics card resources; and DoB can schedule CPU and sound card resources.
[0112] Therefore, for a video translation queue containing the message "playing a scene of hydrogen burning in oxygen", the translation container DoA should be invoked; while for a text-to-image translation queue containing the message "next, display the chemical formula for hydrogen burning in oxygen", the translation container DoC should be invoked.
[0113] Obviously, Figures 5-6 The hardware architecture can be used to execute Figures 1-3 For example, in each of the aforementioned steps or principles, the sequence grouping unit can re-establish the preset grouping time threshold corresponding to each grouping queue based on the number of sequences to be translated contained in each grouping queue, the number of translation modes of the sequences to be translated contained in the grouping queue, the sequence generation time of the sequences to be translated contained in the grouping queue, and a preset grouping time reference value.
[0114] Input: the number K of the sequences to be translated contained in the group queue, the number M of the translation modes of the sequences to be translated contained in the group queue, and the set of sequence generation times SetT of the sequences to be translated contained in the group queue;
[0115] Output: The preset grouping time threshold GroupThreshold corresponding to the grouped queue satisfies the following conditions:
[0116]
[0117] Where Std is the preset grouping time base value.
[0118] In practice, the preset grouping time threshold for each group queue is a baseline value Std. First, the initial elements of the group queue are determined using the baseline value Std. Then, it is calculated whether the preset grouping time threshold of the group queue conforms to the above formula. If it does not conform, the number of elements in the group queue is adjusted (by adding or deleting elements) until the preset grouping time threshold of the group queue conforms to the above formula.
[0119] As can be seen, the advantages of the present invention compared to the prior art include at least the following:
[0120] (1) By using a pre-established multimodal keyword database, after receiving the sequence to be translated, the multimodal keywords and sequence generation time are first identified; then, all the sequences to be translated are added to the corresponding group queue based on the multimodal keywords and sequence generation time, which can realize group translation and scheduling.
[0121] (2) For each group, the corresponding translation container is scheduled to translate, so that the schedulable resources and processes of the translation container into which the group enters match the translation modality of the sequence to be translated itself, avoiding frequent process switching and resource type switching, and reducing the degree of system resource fragmentation.
[0122] (3) The difference in generation time between any two sequences to be translated contained in each group queue is less than the preset grouping time threshold, so that the time when the translation sequence enters the group queue and the time when the translation result is output are synchronized, thus avoiding the situation of sequence disorder.
[0123] (4) Different preset group time thresholds correspond to different group queues, which can ensure that the size and generation time difference of each group queue are consistent with the actual changes of the current group, thereby further ensuring that the number of translation containers corresponding to the group queue is more reasonable, and there will be no data transmission delay or translation waiting, thus improving the user experience.
[0124] Of course, it is understood that each embodiment of the present invention can achieve one of the effects on its own, and the combination of multiple embodiments of the present invention can achieve all the above effects. However, it is not required that each embodiment of the present invention achieve all the above advantages and effects, because each embodiment of the present invention can constitute a separate technical solution and make one or more contributions to the prior art.
[0125] For any module structures not specifically defined in this invention, the existing technical specifications shall prevail. The existing technical specifications mentioned in the foregoing background and specific embodiments sections are considered part of this invention and are used to understand the meaning of certain technical features or parameters. The scope of protection of this invention is determined by the actual contents of the claims.
Claims
1. A multimodal grouping translation method based on keyword recognition, characterized in that, The method includes: The sequence to be translated identification step is used to receive the sequence to be translated and identify the multimodal keywords and sequence generation time; the multimodal keywords are used to characterize the translation modality of the translation result of the sequence to be translated, and the translation modality includes at least two combinations of plain text translation, plain speech translation, image-text translation, and video translation; The process of grouping sequences to be translated is as follows: Each sequence to be translated is added to a corresponding grouping queue based on the multimodal keywords and the sequence generation time; the translation modalities of the sequences to be translated contained in each grouping queue are the same or partially the same. Group translation steps: Determine the translation modality of the translation result of the sequence to be translated contained in each group queue; determine the corresponding translation container based on the translation modality; for each group, schedule the corresponding translation container for translation; Among them, the types of physical resources that can be scheduled by the virtual translator corresponding to the translation container of different translation modalities are different; The difference in generation time between any two sequences to be translated contained in each group queue is less than a preset grouping time threshold, and the preset grouping time thresholds are different for different group queues.
2. The multimodal grouping translation method based on keyword recognition as described in claim 1, characterized in that: The method is executed in parallel by a group translation terminal, which is configured with multiple translation containers. Each translation container runs a virtual translator. Different virtual translators share the physical resources of the group translation terminal in a time-sharing manner or in parallel real-time group sharing of the physical resources of the group translation terminal.
3. The multimodal grouping translation method based on keyword recognition as described in claim 1, characterized in that: A multimodal keyword database is pre-constructed, which stores multiple multimodal keywords; the sequence identification step identifies the multimodal keywords in the sequence to be translated using the pre-constructed multimodal keyword database.
4. The multimodal grouping translation method based on keyword recognition as described in claim 2, characterized in that: The physical resources include a combination of at least two of the following: GPU resources, video memory resources, sound card resources, and CPU resources.
5. The multimodal grouping translation method based on keyword recognition as described in claim 1, characterized in that: In the step of grouping the sequences to be translated, sequences whose translation modalities are video translation and text-image translation are added to the same grouping queue.
6. The multimodal grouping translation method based on keyword recognition as described in claim 1, characterized in that: The preset grouping time threshold for each grouping queue is determined by the number of sequences to be translated contained in the grouping queue, the number of translation modes of the sequences to be translated contained in the grouping queue, and the sequence generation time of the sequences to be translated contained in the grouping queue.
7. A multimodal group translation system based on keyword recognition, the system comprising a multimodal keyword database, characterized in that: The multimodal keyword database pre-stores multiple multimodal keywords, which are used to characterize the translation modality of the translation result of the sequence to be translated. The translation modality includes at least two combinations of plain text translation, plain speech translation, image-text translation, and video translation. The system also includes: The sequence to be translated identification unit is used to receive the sequence to be translated and identify the multimodal keywords and sequence generation time in it through a pre-built multimodal keyword database. Grouping of sequences to be translated: Each sequence to be translated is added to the corresponding grouping queue based on the multimodal keywords and sequence generation time; the translation modalities of the sequences to be translated contained in each grouping queue are the same or partially the same; Group Translation Unit: Determines the translation modality of the translation result of the sequence to be translated contained in each group queue; determines the corresponding translation container based on the translation modality; and schedules the corresponding translation container for translation for each group. Among them, the types of physical resources that can be scheduled by the virtual translator corresponding to the translation container of different translation modalities are different; The difference in generation time between any two sequences to be translated contained in each group queue is less than a preset grouping time threshold.
8. The multimodal group translation system based on keyword recognition as described in claim 7, characterized in that: The translation container runs on a grouped translation terminal. Each translation container runs a virtual translator. Different virtual translators share the physical resources of the grouped translation terminal in a time-sharing manner or in parallel real-time grouped sharing of the physical resources of the grouped translation terminal.
9. A multimodal group translation system based on keyword recognition as described in claim 8, characterized in that: The physical resources include a combination of at least two of the following: GPU resources, video memory resources, sound card resources, and CPU resources. The group translation unit determines the translation modality of the sequence to be translated contained in each group queue, and determines the translation container corresponding to each group queue based on the translation modality; If the translation modal of the sequence to be translated contained in the first group queue includes text-image translation and / or video translation, then the virtual translator running the translation container corresponding to the first group queue can schedule the GPU and graphics card resources of the group translation terminal.
10. A multimodal group translation system based on keyword recognition as described in claim 7, characterized in that: Different preset grouping time thresholds correspond to different grouping queues.