A multi-terminal multi-modal collaboration supported multi-lingual intelligent translation system

Abstract

The application discloses a kind of multilingual intelligent translation system supporting multi-terminal multimodal cooperation, belong to speech recognition, multilingual translation and multi-terminal man-machine interaction technical field, including multiple mobile terminals, multiple mobile terminal each built-in voice acquisition module, speech recognition module, translation module, speech synthesis module and display module, the local voice is acquired based on microphone technology in mobile terminal, and the current language category is identified, and then real-time voice is converted into text, system supports double mobile terminal pairing communication and multiple mobile terminal networking communication, low delay synchronization is realized by Bluetooth, Wi-Fi, local protocol or other point-to-point mode, mobile terminal is based on interactive rhythm control logic, in turn enter speech / reception state, form natural interaction closed loop, in the application, it can realize face-to-face double end or the language automatic identification between multiple ends, instant translation, two-way voice broadcast and synchronous display, to meet the instant communication demand in real scene.

Description

Technical Field

[0001] This invention belongs to the fields of speech recognition, multilingual translation and multi-terminal human-computer interaction technology, and particularly relates to a multilingual intelligent translation system that supports multi-terminal and multimodal collaboration. Background Technology

[0002] Current mainstream translation technologies still rely on a single device operating independently as their core architecture, which exposes significant limitations when dealing with complex interpersonal communication scenarios. Taking high-dynamic scenarios such as law enforcement, port scheduling, and airport emergencies as examples, their two-way or multi-party cross-language interaction needs exhibit three main characteristics:

[0003] The flow of language is highly dynamic, such as multiple roles taking turns speaking and impromptu interruptions;

[0004] The information structure is non-linear, with multiple complex structures involving topic jumps and semantic nesting;

[0005] The interaction rhythm is highly real-time, requiring sub-second response latency. However, existing solutions are mostly designed based on the "single device-single dialogue" mode, which faces three technical bottlenecks when dealing with such scenarios.

[0006] In contrast, this invention completely solves the problem of existing systems being unable to achieve low-latency synchronization across devices by defining a multi-terminal synchronization and collaboration protocol. Based on this, this invention designs a multilingual intelligent translation system that supports multi-terminal, multimodal collaboration to address the aforementioned problems. Summary of the Invention

[0007] The purpose of this invention is to address the lack of effective support in existing solutions, particularly in face-to-face communication, where inconsistencies in language direction, disordered speaking order, and fragmented semantic rhythm severely impact translation quality and user experience. Therefore, this invention proposes a multilingual intelligent translation system that supports multi-terminal, multimodal collaboration.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A multilingual intelligent translation system supporting multi-terminal and multimodal collaboration includes multiple mobile terminals, each of which has a built-in voice acquisition module, voice recognition module, translation module, voice synthesis module, and display module.

[0010] The mobile terminal uses microphone technology to collect local speech, identifies the current language, and then converts the speech into text in real time.

[0011] The system supports dual mobile terminal pairing communication and multi-mobile terminal networking communication, and achieves low-latency synchronization through Bluetooth, Wi-Fi, local area protocols or other point-to-point methods;

[0012] Mobile terminals, based on interactive rhythm control logic, alternately enter speaking / receiving states, forming a natural interactive closed loop;

[0013] The system expands the role-driven content view and emotion scene recognition corpus to further improve scene adaptability and semantic accuracy. Compared with existing technologies, this system introduces a unified multi-terminal calling process at the protocol level, realizing end-to-end collaboration between the end and the cloud.

[0014] As a further description of the above technical solution:

[0015] The input terminal of the translation engine module is electrically connected to the output terminals of the speech recognition module and the mixed input processing module, respectively. The input terminal of the mixed input processing module is electrically connected to the output terminals of the text input module and the hidden mode detection and switching module, respectively. The input terminal of the speech recognition module is electrically connected to the output terminals of the speech input module and the text input module, respectively. The output terminal of the text input module is electrically connected to the input terminal of the translation engine module. The input terminals of both the speech input module and the text input module are electrically connected to the output terminal of the operation prompt and feedback module.

[0016] The input end of the translation engine module is electrically connected to the output end of the multilingual language resource database management system and the scene adaptation function module, respectively. The output ends of the translation engine module, the scene adaptation function module and the operation prompt and feedback module are all electrically connected to the input end of the output module.

[0017] The input terminal of the output module is electrically connected to the output terminals of the multi-terminal synchronization module and the translation module, respectively. The input terminals of the multi-terminal synchronization module and the translation module are both electrically connected to the output terminal of the user identity and permission management module.

[0018] As a further description of the above technical solution:

[0019] The input end of the output module is electrically connected to the output end of the PHP text translator. The input end of the PHP text translator is electrically connected to the output end of the mixed input module. The input end of the mixed input module is electrically connected to the output ends of multiple PHP backend servers. The input ends of the multiple PHP backend servers are all electrically connected to the output end of the WeChat mini program. The input end of the WeChat mini program is electrically connected to the output end of the login module.

[0020] As a further description of the above technical solution:

[0021] The voice input module is used to collect and input multilingual voice data. The text input module supports direct text input for translation or as a substitute for voice. The voice recognition module converts voice input into text information. The translation engine module performs multilingual translation of the recognized text. The output module supports displaying translation results on the screen, through headphones, or via external speaker. The mixed input processing module processes both voice and text input simultaneously and automatically adapts to the mixed input mode. The multi-terminal synchronization module enables real-time collaboration among multiple mobile terminals and synchronized push of translation results. The hidden mode detection and switching module automatically detects continuous input behavior and activates the hidden mixed input mode without requiring manual switching by the user. The translation module converts text into the target language and provides voice playback via the voice module. The user identity and permission management module classifies permissions and authenticates users on different terminals. The scenario adaptation module automatically optimizes the interface and translation strategy according to the application scenario. The operation prompt and feedback module provides real-time prompts, error feedback, and confirmation mechanisms to ensure a smooth interactive experience.

[0022] As a further description of the above technical solution:

[0023] The multilingual language resource database management system includes a user management module, a terminology management module, a search module, a self-learning management module, and a resource database maintenance module;

[0024] The user management module clearly defines three user types: terminology administrators, review experts, and system administrators. Different types of users perform corresponding operations within the system based on their assigned permissions. Specifically, terminology administrators have comprehensive management permissions for terms, including the creation, editing, and deletion of terms, while review experts only have the permission to review term-related content to ensure the accuracy and standardization of terms.

[0025] As a further description of the above technical solution:

[0026] The terminology management module is scientifically and systematically divided into two core sections: standard terminology management and extended terminology management. This module undertakes the crucial task of language resource construction and maintenance. Functionally, it focuses on the comprehensive collection of terms and phrases, ensuring the breadth and accuracy of terminology sources through multi-channel and multi-method collection strategies. Furthermore, relying on a professional translation team and advanced translation technology, it accurately translates the collected terms and phrases, providing a solid guarantee for their use in different language environments. Simultaneously, the terminology management module possesses efficient input capabilities, enabling the rapid and accurate input of translated terms and phrases into the system database.

[0027] As a further description of the above technical solution:

[0028] The retrieval module includes a customer management submodule, a customer job management submodule, and a task management submodule. The customer management submodule enables searching for customer-related information and querying, modifying, and deleting customer information. The customer job management submodule enables searching for customer jobs and managing job assignments. The task management submodule is used to query task information.

[0029] As a further description of the above technical solution:

[0030] The self-learning management module is mainly used to re-examine terms and phrases that have not been retrieved. In the self-learning management module, you can view the content that needs to be self-learned. Review experts can use the self-learning management module to review, add, view, modify and delete new terms, and also view the progress status of the expert's review of terms.

[0031] As a further description of the above technical solution:

[0032] The resource library maintenance module, which is a multilingual language resource library maintenance module, includes a data backup submodule and a system log submodule. The system administrator can use the data backup submodule to back up, restore, and query the language resource library data, and use the system log submodule to view and search the system logs.

[0033] As a further description of the above technical solution:

[0034] The core technology of the translation engine module is divided into three key parts: preprocessing for text sequences, speech translation combined with a multilingual language resource database management system, and postprocessing for text sequences.

[0035] The translation engine module employs a dynamic time warping algorithm to accurately measure time series. This algorithm analyzes the characteristics of different time series, calculates their similarity, and adaptively warps the time series based on the calculation results to achieve efficient response. Let there be time series X and time series Y, where time series X has length i and time series Y has length j. Thus, an i×j distance matrix is ​​obtained. In the distance matrix, (a, b) represents aligning point a in time series X with point b in time series Y. The dynamic time warping algorithm finds the optimal solution for the curved path to ensure minimum similarity. Let L be the curved path corresponding to the time series, and its corresponding set is represented as:

[0036] L={(a1,b1),(a2,b2),…,(a n ,b n )}

[0037] In the formula, the bending path L should satisfy the response constraints, and the constraints for continuity are expressed as follows:

[0038]

[0039] Meanwhile, the constraints established for monotonicity are:

[0040]

[0041] The following constraints are established regarding convenience:

[0042]

[0043] Therefore, the minimum distance for dynamic time warping is calculated as follows:

[0044]

[0045] In the formula, T represents the path constraint condition. Representing the midpoint of different time series With point The distance between them;

[0046] The dynamic time series algorithm error correction technology relies on an advanced algorithm architecture to build a core processing mechanism, focusing on the accurate recognition and efficient processing of spoken speech. With the help of a carefully designed speech acquisition program, it can accurately capture speech signals with errors. Then, it uses targeted error correction strategies to analyze and correct the acquired erroneous speech in detail, thereby ensuring that mobile terminals achieve higher accuracy and reliability in the process of speech translation.

[0047] When the dataset is large, similarity is obtained by weighting the extreme value sequences to achieve optimization;

[0048] For a time series X = (a1, a2, ..., a... j ,…,a n The conditions for determining local extrema are as follows:

[0049]

[0050] By organically integrating local extreme points with the start and end points of the time series, a simplified set of key points is constructed. Based on this, the original time series is subjected to dimensionality reduction. This dimensionality reduction strategy significantly reduces the computational complexity of the time series while preserving the overall characteristics and trends of the original series to the greatest extent. It ensures that the dimensionality-reduced series can completely and accurately reflect the data characteristics of the original time series, providing a reliable basis for subsequent data analysis and processing.

[0051] To ensure that the alignment of different time series minimizes the deviation between different series points, the number of singularities is reduced by weighting the extreme values. The weights are calculated as follows:

[0052]

[0053] In the formula, M max Let z(a,b) represent the expected maximum value of the function, α represent the weight parameters, z(a,b) represent the phase difference, and p(a,b) represent the type of sequence point pair. n The midpoint of the sequence is represented by , and S represents the length of the sequence.

[0054] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0055] Compared with traditional single-terminal translation systems, this invention has significant advantages in low latency, multimodal operation, and multi-terminal collaboration.

[0056] 1. This invention enables automatic language recognition, real-time translation, two-way voice broadcasting, and synchronous display between face-to-face two- or multi-terminal devices, thereby meeting the real-time communication needs in real-world scenarios and improving efficiency and accuracy.

[0057] 2. This invention can significantly improve the efficiency of face-to-face translation, and is particularly suitable for multiple high-frequency real-world scenarios such as law enforcement, ports, customs clearance, complaints, and services. It supports two-person or multi-party interaction, has the ability to switch language directions, adapts to complex language environments, has a clear technical solution, a reasonable structure, is easy to implement, has broad industry promotion value, supports industry customization, access control and device binding, and is convenient for standardized deployment by government agencies and enterprises.

[0058] 3. In this invention, relying on the powerful data processing capabilities of the PHP backend server and combined with the rich and diverse tag system of WeChat Mini Programs, a multimodal input system has been successfully built. This multimodal input system can not only efficiently process text input, but also accurately recognize image information and smoothly receive voice content, realizing the organic integration of text, image, and voice input methods. At the same time, with the help of advanced translation algorithms and language models, it provides users with real-time and accurate multilingual translation services. This innovative solution effectively solves the prominent problems of existing translation software, such as cumbersome and complicated installation process, single and fixed input methods, and limited language coverage, significantly improving the user experience and translation efficiency.

[0059] 4. In this invention, the establishment of the multilingual language resource database management system fills the gap in the development of multilingual related software industry, and significantly improves the system's universality, interoperability and scalability. Attached Figure Description

[0060] Figure 1This is a diagram illustrating the architecture of a multilingual intelligent translation system supporting multi-terminal, multi-modal collaboration, as proposed in this invention.

[0061] Figure 2 This is a schematic diagram of multi-terminal collaborative communication in a multilingual intelligent translation system that supports multi-terminal and multimodal collaboration, as proposed in this invention.

[0062] Figure 3 This is a schematic diagram of the turn-to-speak mode of a multilingual intelligent translation system supporting multi-terminal and multi-modal collaboration proposed in this invention;

[0063] Figure 4 This is a schematic diagram of a hybrid speech mode for a multilingual intelligent translation system that supports multi-terminal and multimodal collaboration, as proposed in this invention.

[0064] Figure 5 This is a schematic diagram of a multi-terminal collaborative meeting mode for a multilingual intelligent translation system supporting multi-terminal and multi-modal collaboration proposed in this invention;

[0065] Figure 6 This is a block diagram of a multilingual intelligent translation system supporting multi-terminal and multimodal collaboration proposed in this invention.

[0066] Figure 7 This is a software architecture diagram of a multilingual real-time translation system based on multimodal input, which supports multi-terminal and multimodal collaboration and is proposed in this invention.

[0067] Figure 8 This is a block diagram of a multilingual intelligent translation system and a multilingual language resource database management system that supports multi-terminal, multi-modal collaboration, as proposed in this invention. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] Please see the appendix Figure 1 - Appendix Figure 8 The present invention provides a technical solution: a multilingual intelligent translation system that supports multi-terminal and multimodal collaboration, including multiple mobile terminals, each of which has a built-in voice acquisition module, voice recognition module, translation module, voice synthesis module and display module;

[0070] The mobile terminal uses microphone technology to collect local speech, identifies the current language, and then converts the speech into text in real time.

[0071] The system supports dual mobile terminal pairing communication and multi-mobile terminal networking communication, and achieves low-latency synchronization through Bluetooth, Wi-Fi, local area protocols or other point-to-point methods;

[0072] Based on interactive rhythm control logic, mobile terminals take turns entering the speaking / receiving state, forming a natural interactive closed loop. The multi-terminal synchronous calling process of this system is as follows: audio acquisition → recognition → translation → multi-terminal distribution → status confirmation, with a synchronization delay of ≤100ms.

[0073] The system is used to expand the role-driven content view and the emotion scene recognition corpus to further improve scene adaptability and semantic accuracy.

[0074] Specifically, the input end of the translation engine module is electrically connected to the output ends of the speech recognition module and the mixed input processing module, respectively. The input end of the mixed input processing module is electrically connected to the output ends of the text input module and the hidden mode detection and switching module, respectively. The input end of the speech recognition module is electrically connected to the output ends of the speech input module and the text input module, respectively. The output end of the text input module is electrically connected to the input end of the translation engine module. The input ends of both the speech input module and the text input module are electrically connected to the output end of the operation prompt and feedback module.

[0075] The input end of the translation engine module is electrically connected to the output end of the multilingual language resource database management system and the scene adaptation function module, respectively. The output ends of the translation engine module, the scene adaptation function module and the operation prompt and feedback module are all electrically connected to the input end of the output module.

[0076] The input terminal of the output module is electrically connected to the output terminals of the multi-terminal synchronization module and the translation module, respectively. The input terminals of the multi-terminal synchronization module and the translation module are both electrically connected to the output terminal of the user identity and permission management module.

[0077] Specifically, the input end of the output module is electrically connected to the output end of the PHP text translator, the input end of the PHP text translator is electrically connected to the output end of the mixed input module, the input end of the mixed input module is electrically connected to the output ends of multiple PHP backend servers, the input ends of the multiple PHP backend servers are all electrically connected to the output end of the WeChat mini program, and the input end of the WeChat mini program is electrically connected to the output end of the login module.

[0078] Specifically, the voice input module is used to collect and input multilingual voice data; the text input module supports direct text input for translation or as a substitute for voice; the voice recognition module converts voice input into text information; the translation engine module performs multilingual translation of the recognized text; the output module supports displaying translation results on the screen, through headphones, or via external speaker; the mixed input processing module processes both voice and text input simultaneously and automatically adapts to the mixed input mode; the multi-terminal synchronization module enables real-time collaboration among multiple mobile terminals and synchronized push of translation results; the hidden mode detection and switching module automatically detects continuous input behavior and activates the hidden mixed input mode without requiring manual switching by the user; the translation module converts text into the target language and provides voice playback through the voice module; the user identity and permission management module classifies permissions and authenticates users on different terminals; the scenario adaptation module automatically optimizes the interface and translation strategy according to the application scenario; and the operation prompts and feedback module provides real-time prompts, error feedback, and confirmation mechanisms to ensure a smooth interactive experience.

[0079] Specifically, the multilingual language resource database management system includes a user management module, a terminology management module, a search module, a self-learning management module, and a resource database maintenance module;

[0080] The user management module clearly defines three user types: terminology administrator, review expert, and system administrator. Different types of users perform corresponding operations within the system based on their assigned permissions. Specifically, terminology administrators have comprehensive management permissions for terms, including the creation, editing, and deletion of terms, while review experts only have the permission to review term-related content to ensure the accuracy and standardization of terms.

[0081] The system employs a hierarchical management model of "user-user group-role-permission" to provide refined management of users in the language resource library. This model clearly defines the operational scope of each user, ensuring the security and stability of system data. In terms of user management module functionality, it features comprehensive user permission management capabilities. Administrators can flexibly adjust user permissions, including adding permissions for new users, deleting unnecessary permissions, modifying existing permission settings, and viewing detailed user permission information. Furthermore, the user management module supports keyword-based user information retrieval, allowing administrators to quickly locate specific users and improve management efficiency.

[0082] Specifically, the terminology management module is scientifically and systematically divided into two core sections: standard terminology management and extended terminology management. This module undertakes the crucial task of language resource construction and maintenance. Functionally, it focuses on the comprehensive collection of terms and phrases, ensuring the breadth and accuracy of terminology sources through multi-channel and multi-method collection strategies. Furthermore, relying on a professional translation team and advanced translation technology, it accurately translates the collected terms and phrases, providing a solid guarantee for their use in different language environments. Simultaneously, the terminology management module possesses efficient input capabilities, enabling the rapid and accurate input of translated terms and phrases into the system database.

[0083] The terminology management module provides comprehensive terminology database maintenance functions. For terms already added to the database, it supports flexible addition operations based on actual needs, continuously enriching the content of the terminology database. Outdated, incorrect, or no longer applicable terms are deleted in a timely manner to ensure the timeliness and accuracy of the terminology database. Furthermore, the module allows users to quickly locate target terms through convenient search functions for further viewing and modification operations, thereby achieving dynamic management and optimization of the terminology database.

[0084] Specifically, the retrieval module includes a customer management submodule, a customer job management submodule, and a task management submodule. The customer management submodule enables searching for customer-related information and querying, modifying, and deleting customer information. The customer job management submodule enables searching for customer jobs and managing job assignments. The task management submodule is used to query task information.

[0085] Specifically, the self-learning management module is mainly used to re-examine terms and phrases that have not been retrieved. In the self-learning management module, the content that needs to be self-learned can be viewed. Review experts can use the self-learning management module to review, add, view, modify and delete new terms, and can also use it to view the progress status of the expert's review of terms.

[0086] Specifically, the resource library maintenance module, the multilingual language resource library maintenance module, has a data backup submodule and a system log submodule. The system administrator can back up, restore and query the language resource library data through the data backup submodule, and view and search the system log in the system log submodule.

[0087] Specifically, the core technology of the translation engine module is divided into three key parts: preprocessing for text sequences, speech translation in conjunction with a multilingual language resource database management system, and postprocessing for text sequences.

[0088] The translation engine module employs a dynamic time warping algorithm to accurately measure time series. This algorithm analyzes the characteristics of different time series, calculates their similarity, and adaptively warps the time series based on the calculation results to achieve efficient response. Let there be time series X and time series Y, where time series X has length i and time series Y has length j. Thus, an i×j distance matrix is ​​obtained. In the distance matrix, (a, b) represents aligning point a in time series X with point b in time series Y. The dynamic time warping algorithm finds the optimal solution for the curved path to ensure minimum similarity. Let L be the curved path corresponding to the time series, and its corresponding set is represented as:

[0089] L={(a1,b1),(a2,b2),…,(a n ,b n )}

[0090] In the formula, the bending path L should satisfy the response constraints, and the constraints for continuity are expressed as follows:

[0091]

[0092] Meanwhile, the constraints established for monotonicity are:

[0093]

[0094] The following constraints are established regarding convenience:

[0095]

[0096] Therefore, the minimum distance for dynamic time warping is calculated as follows:

[0097]

[0098] In the formula, T represents the path constraint condition. Representing the midpoint of different time series With point The distance between them;

[0099] The dynamic time series algorithm error correction technology relies on an advanced algorithm architecture to build a core processing mechanism, focusing on the accurate recognition and efficient processing of spoken speech. With the help of a carefully designed speech acquisition program, it can accurately capture speech signals with errors. Then, it uses targeted error correction strategies to analyze and correct the acquired erroneous speech in detail, thereby ensuring that mobile terminals achieve higher accuracy and reliability in the process of speech translation.

[0100] When the dataset is large, similarity is obtained by weighting the extreme value sequences to achieve optimization;

[0101] For a time series X = (a1, a2, ..., a... j ,…,a n The conditions for determining local extrema are as follows:

[0102]

[0103] By organically integrating local extreme points with the start and end points of the time series, a simplified set of key points is constructed. Based on this, the original time series is subjected to dimensionality reduction. This dimensionality reduction strategy significantly reduces the computational complexity of the time series while preserving the overall characteristics and trends of the original series to the greatest extent. It ensures that the dimensionality-reduced series can completely and accurately reflect the data characteristics of the original time series, providing a reliable basis for subsequent data analysis and processing.

[0104] To ensure that the alignment of different time series minimizes the deviation between different series points, the number of singularities is reduced by weighting the extreme values. The weights are calculated as follows:

[0105]

[0106] In the formula, M max Let z(a,b) represent the expected maximum value of the function, α represent the weight parameters, z(a,b) represent the phase difference, and p(a,b) represent the type of sequence point pair. n The midpoint of the sequence is represented by , and S represents the length of the sequence.

[0107] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multilingual intelligent translation system supporting multi-terminal, multimodal collaboration, comprising multiple mobile terminals, characterized in that, Each of the aforementioned mobile terminals has a built-in voice acquisition module, voice recognition module, translation module, voice synthesis module, and display module; The mobile terminal uses microphone technology to collect local speech, identifies the current language, and then converts the speech into text in real time. The system supports dual mobile terminal pairing communication and multi-mobile terminal networking communication, and achieves low-latency synchronization through Bluetooth, Wi-Fi, local area protocols or other point-to-point methods; Mobile terminals, based on interactive rhythm control logic, alternately enter speaking / receiving states, forming a natural interactive closed loop; The system is used to expand the role-driven content view and emotion scene recognition corpus to further improve scene adaptability and semantic fidelity. The system supports cross-regional terminal communication based on the Internet.

2. The multilingual intelligent translation system supporting multi-terminal, multi-modal collaboration according to claim 1, characterized in that, The input terminal of the translation engine module is electrically connected to the output terminals of the speech recognition module and the mixed input processing module, respectively. The input terminal of the mixed input processing module is electrically connected to the output terminals of the text input module and the hidden mode detection and switching module, respectively. The input terminal of the speech recognition module is electrically connected to the output terminals of the speech input module and the text input module, respectively. The output terminal of the text input module is electrically connected to the input terminal of the translation engine module. The input terminals of both the speech input module and the text input module are electrically connected to the output terminal of the operation prompt and feedback module. The input end of the translation engine module is electrically connected to the output end of the multilingual language resource database management system and the scene adaptation function module, respectively. The output ends of the translation engine module, the scene adaptation function module and the operation prompt and feedback module are all electrically connected to the input end of the output module. The input terminal of the output module is electrically connected to the output terminals of the multi-terminal synchronization module and the translation module, respectively. The input terminals of the multi-terminal synchronization module and the translation module are both electrically connected to the output terminal of the user identity and permission management module.

3. A multilingual intelligent translation system supporting multi-terminal, multi-modal collaboration according to claim 2, characterized in that, The input end of the output module is electrically connected to the output end of the PHP text translator. The input end of the PHP text translator is electrically connected to the output end of the mixed input module. The input end of the mixed input module is electrically connected to the output ends of multiple PHP backend servers. The input ends of the multiple PHP backend servers are all electrically connected to the output end of the WeChat mini program. The input end of the WeChat mini program is electrically connected to the output end of the login module. The local communication module is extended to support low-latency synchronization with remote terminals via public network protocols.

4. A multilingual intelligent translation system supporting multi-terminal, multi-modal collaboration according to claim 3, characterized in that, The voice input module is used to collect and input multilingual voice data. The text input module supports direct text input for translation or as a substitute for voice. The voice recognition module converts voice input into text information. The translation engine module performs multilingual translation of the recognized text. The output module supports displaying translation results on the screen, through headphones, or via external speaker. The mixed input processing module processes both voice and text input simultaneously and automatically adapts to the mixed input mode. The multi-terminal synchronization module enables real-time collaboration among multiple mobile terminals and synchronized push of translation results. The hidden mode detection and switching module automatically detects continuous input behavior and activates the hidden mixed input mode without requiring manual switching by the user. The translation module converts text into the target language and provides voice playback through the voice module. The user identity and permission management module classifies permissions and authenticates users on different terminals. The scenario adaptation module automatically optimizes the interface and translation strategy according to the application scenario. The operation prompt and feedback module provides real-time prompts, error feedback, and confirmation mechanisms to ensure a good interactive experience. The rhythm synchronization unit uses a specified data packet format, including text identifiers, rhythm parameters, and user identifiers, to achieve cross-platform interoperability.

5. A multilingual intelligent translation system supporting multi-terminal, multi-modal collaboration according to claim 4, characterized in that, The multilingual language resource database management system includes a user management module, a terminology management module, a search module, a self-learning management module, and a resource database maintenance module; The user management module clearly defines three user types: terminology administrators, review experts, and system administrators. Different user types perform corresponding operations within the system based on their assigned permissions. Specifically, terminology administrators have comprehensive management permissions for terms, including creation, editing, and deletion. Review experts only have permissions to review terminology-related content to ensure the accuracy and standardization of terms. The learning progress management module also generates and records timestamps on the learning behavior chain to ensure data consistency across all endpoints.

6. A multilingual intelligent translation system supporting multi-terminal, multi-modal collaboration according to claim 5, characterized in that, The terminology management module is scientifically and systematically divided into two core sections: standard terminology management and extended terminology management. This module undertakes the crucial task of language resource construction and maintenance. Functionally, it focuses on the comprehensive collection of terms and phrases, ensuring the breadth and accuracy of terminology sources through multi-channel and multi-method collection strategies. Furthermore, relying on a professional translation team and advanced translation technology, it accurately translates the collected terms and phrases, providing a solid guarantee for their use in different language environments. Simultaneously, the terminology management module possesses efficient input capabilities, enabling the rapid and accurate input of translated terms and phrases into the system database.

7. A multilingual intelligent translation system supporting multi-terminal, multi-modal collaboration according to claim 6, characterized in that, The retrieval module includes a customer management submodule, a customer job management submodule, and a task management submodule. The customer management submodule enables searching for customer-related information and querying, modifying, and deleting customer information. The customer job management submodule enables searching for customer jobs and managing job assignments. The task management submodule is used to query task information.

8. A multilingual intelligent translation system supporting multi-terminal, multi-modal collaboration according to claim 7, characterized in that, The self-learning management module is mainly used to re-examine terms and phrases that have not been retrieved. In the self-learning management module, you can view the content that needs to be self-learned. Review experts can use the self-learning management module to review, add, view, modify and delete new terms, and also view the progress status of the expert's review of terms.

9. A multilingual intelligent translation system supporting multi-terminal, multi-modal collaboration according to claim 8, characterized in that, The resource library maintenance module, which is a multilingual language resource library maintenance module, includes a data backup submodule and a system log submodule. The system administrator can use the data backup submodule to back up, restore, and query the language resource library data, and use the system log submodule to view and search the system logs.

10. A multilingual intelligent translation system supporting multi-terminal, multi-modal collaboration according to claim 9, characterized in that, The core technology of the translation engine module is divided into three key parts: preprocessing for text sequences, speech translation combined with a multilingual language resource database management system, and postprocessing for text sequences. The translation engine module employs a dynamic time warping algorithm to accurately measure time series. This algorithm analyzes the characteristics of different time series, calculates their similarity, and adaptively warps the time series based on the calculation results to achieve efficient response. Let there be time series X and time series Y, where time series X has length i and time series Y has length j. Thus, an i×j distance matrix is ​​obtained. In the distance matrix, (a, b) represents aligning point a in time series X with point b in time series Y. The dynamic time warping algorithm finds the optimal solution for the curved path to ensure minimum similarity. Let L be the curved path corresponding to the time series, and its corresponding set is represented as: L={(a1,b1),(a2,b2),…,(a n b n )} In the formula, the bending path L should satisfy the response constraints, and the constraints for continuity are expressed as follows: Meanwhile, the constraints established for monotonicity are: The following constraints are established regarding convenience: Therefore, the minimum distance for dynamic time warping is calculated as follows: In the formula, T represents the path constraint condition. Representing the midpoint of different time series With point The distance between them; The dynamic time series algorithm error correction technology relies on an advanced algorithm architecture to build a core processing mechanism, focusing on the accurate recognition and efficient processing of spoken speech. With the help of a carefully designed speech acquisition program, it can accurately capture speech signals with errors. Then, it uses targeted error correction strategies to analyze and correct the acquired erroneous speech in detail, thereby ensuring that mobile terminals achieve higher accuracy and reliability in the process of speech translation. When the dataset is large, similarity is obtained by weighting the extreme value sequences to achieve optimization; For a time series X = (a1, a2, ..., a... j ,…,a n The conditions for determining local extrema are as follows: By organically integrating local extreme points with the start and end points of the time series, a simplified set of key points is constructed. Based on this, the original time series is subjected to dimensionality reduction. This dimensionality reduction strategy significantly reduces the computational complexity of the time series while preserving the overall characteristics and trends of the original series to the greatest extent. It ensures that the dimensionality-reduced series can completely and accurately reflect the data characteristics of the original time series, providing a reliable basis for subsequent data analysis and processing. To ensure that the alignment of different time series minimizes the deviation between different series points, the number of singularities is reduced by weighting the extreme values. The weights are calculated as follows: In the formula, M max Let z(a,b) represent the expected maximum value of the function, α represent the weight parameters, z(a,b) represent the phase difference, and p(a,b) represent the type of sequence point pair. n The midpoint of the sequence is represented by , and S represents the length of the sequence.

Images