526 results about "Language translation" patented technology

The invention discloses a sign language translation method and device based on vision and word feature pre-training alignment, and belongs to the technical field of sign language translation.The sign language translation method comprises the steps that vision features of sign language videos are extracted through a vision encoder, and word text features are extracted through a text encoder; performing comparative learning based on vision and word text features to obtain a pre-trained vision encoder; performing word text mask prediction pre-training on the text decoder; constructing the pre-trained visual encoder and the pre-trained text decoder into a sign language recognition model so as to recognize a word text sequence from the sign language video; and accessing the pre-trained large language model in the domain into a sign language recognition model to construct a sign language translation model, and performing combined fine tuning so as to translate the word text sequence into a natural language text. The sign language recognition and translation method can realize more efficient, accurate and reliable sign language recognition and translation, is applied to the fields of intelligent sign language translation, barrier-free communication, sign language education and the like, and provides more accurate and natural language interaction experience for hearing-impaired groups.

The invention relates to the technical field of language translation, in particular to a low-resource language translation method and system based on deconstruction distillation, and the method comprises the steps: obtaining parallel corpus data of a low-resource language and a general language; the method comprises the following steps: constructing a teacher model by taking parallel corpus data as input, constructing a student model trunk based on a pre-trained BERT big language model and optimizing the student model, calculating cross-task attention alignment loss based on the teacher model and the student model, and outputting and executing logits distillation based on the teacher model and the student model. Deployment of low-resource language translation is completed based on the trained student model, a high-quality knowledge migration source is provided for the student model by constructing the BERT teacher model subjected to full-parameter fine adjustment, and meanwhile, the problem of translation precision caused by insufficient low-resource language data is effectively solved by means of dual supervision of cross-task attention alignment and logits distillation.

The invention provides a multi-mode voice interaction method based on a large model, electronic equipment and a storage medium, and the method comprises the steps: carrying out the voice recognition of the voice data of a first language inputted by a user, and obtaining a first language text; performing language translation on the first language text to translate the first language text into a second language text; performing semantic analysis on the second language text to obtain a semantic analysis result; performing multi-modal knowledge graph construction on the semantic analysis result based on a preset large model to obtain an enhanced semantic understanding result; wherein the multi-modal information related to the second language text is fused in the multi-modal knowledge graph; generating a dialogue strategy of the enhanced semantic understanding result; and performing voice synthesis based on the dialogue strategy to generate reply voice data expressed in the first language. According to the method and the device, multi-language interaction is realized, and meanwhile, various modal information is fused.

The invention relates to the technical field of subtitle translation, in particular to a multi-modal time sequence alignment AI video translation method and system, and the method comprises the steps: 1, carrying out the multi-modal analysis of a to-be-translated video, and obtaining audio separation data, voiceprint feature data and visual time sequence data; 2, performing cross-language translation and context optimization on the basis of the voice of the audio separation data to generate a target language text, and synthesizing target language voice retaining the original voice color in combination with the voiceprint feature data and the target language text; generating a mouth shape animation matched with the target language voice based on the lip key point data and the limb action time sequence data; and step 3, performing four-dimensional alignment on the target language voice, the translated text, the mouth shape animation and the limb action sequence through a cross-modal time sequence encoder, and dynamically adjusting the layout of the bilingual subtitles to adapt to a video picture. According to the method and the device, multi-mode synchronization can be taken into consideration during video translation, so that the body actions such as voice, subtitles and mouth shapes are kept aligned.

The invention discloses a multi-language translation method and system, and relates to the field of electrical digital data process.The method comprises the steps that cultural feature information and semantic structure information are obtained; splitting the source language text into a plurality of semantic units based on the semantic structure information; retrieving cultural connotation information of which the association degree with the semantic unit is greater than a preset threshold value; determining that the semantic unit is a culture sensitive semantic unit; extracting a target language expression mode based on the cultural connotation information; performing semantic combination on the target language expression mode and standard translation results of other semantic units except the culture sensitive semantic unit in the source language text to obtain a translation result; judging whether the semantic coherence of the translation result meets a preset condition or not; if the preset condition is met, outputting a translation result; and if the preset condition is not met, re-executing the step of semantic combination. The accuracy of the cultural connotation and the emotional color contained in the transmission source language is improved, and then the cultural adaptability of the translation result is improved.

The invention relates to the technical field of language translation, in particular to a translation precision optimization method and system based on artificial intelligence, and the method comprises the steps: obtaining a source language text, carrying out the phrasing operation on the source language text to obtain a source language sentence sequence of the source text, carrying out the word segmentation operation on each source language sentence in the source language sentence sequence, and carrying out the word segmentation operation on each source language sentence; obtaining a word sequence of the source language sentence; and performing sentiment classification on each source language sentence in the source language sentence sequence based on a pre-trained first sentiment classification model. The sentiment classification is performed on the source language sentences and the target language sentences on the basis of the first sentiment classification model and the second sentiment classification model, and the similarity between the sentences is calculated, so that the sentences after translation are kept consistent in sentiment, the problem of sentiment deviation generated in a machine translation process is favorably avoided, and the translation efficiency is improved. Especially for some texts with strong emotion colors, it is ensured that translated emotion information can be correctly conveyed.

System and techniques to facilitate the translation of a sign language into another language are described herein. A modular architecture may be used in which the output of different classifiers may be used to produce intermediate representations, or final translations, of the sign language. These classifiers may be trained on different types of signs to enhance accuracy while reduce training time and complexity.

The invention provides a sign language translation method and system based on a pre-training diffusion large language model, and belongs to the field of sign language video translation. The method comprises the following steps: preprocessing a video containing sign language actions to obtain a sign language video frame sequence, inputting the sign language video frame sequence into a visual feature extraction network to extract features, and fusing to obtain a time sequence visual fusion feature sequence; giving a text cue word of a sign language translation task, constructing an initial mask sequence for a target translation position, taking the text cue word, the time sequence visual fusion feature sequence and the initial mask sequence as guide conditions, injecting the guide conditions into a diffusion language model, iteratively denoising and predicting lexical elements of a masked position in combination with a diffusion mask mechanism, and obtaining the sign language translation task. A natural language translation sequence is obtained, and sign language translation is completed; wherein when the diffusion language model is trained, through an internal feature alignment mechanism, the guiding effect of guiding conditions on text generation is optimized, so that the accuracy, coherence and robustness of long text translation are improved, and the actual requirements of a barrier-free public service scene are better met.

The invention belongs to the technical field of natural language processing and intelligent knowledge retrieval, and provides an open type question implicit intention recognition method and system based on multi-agent reasoning. According to the technical scheme, multi-language background knowledge is constructed through a multi-language translation agent and an information retrieval agent; based on the constructed multi-language background knowledge, subject analysis is carried out, multi-field expert agent information is generated, and a multi-field expert agent is dynamically constructed; according to the guide problem proposed by the host agent, generating respective implicit intentions and reasoning argument text contents by using the multi-field expert agent according to the guide problem proposed by the host agent; and performing quality evaluation on the implicit intention and the reasoning argumentation text content by utilizing the host agent, and constructing an implicit intention library by taking the speech of each screened expert agent as a node. The defects in the aspects of processing complex semantic understanding and the like are effectively overcome, and the implicit intention in the open type problem is recognized more accurately and comprehensively.

The invention discloses a real-time place name and address multi-language translation system and method based on a neural network model, and relates to the technical field of multi-language translations, and the method comprises the steps: firstly extracting a topological relation of address components through structured analysis, and then carrying out the deep modeling of cross-component context dependence through employing a deep learning-based neural network model, deep semantic representation of the address text is mined through cross-component semantic association learning, and place name disambiguation based on dynamic semantic reasoning is realized based on the deep semantic representation; according to the method, the problem of semantic deviation in complex scenes such as homonymy, different places and different names is solved, the accuracy of judging the ambiguity of geographic entities is remarkably improved, the translation result can dynamically adapt to place name expression habits of different languages, translation ambiguity caused by limitation of a static knowledge base or imperfect rules is effectively eliminated, and the translation efficiency is improved. And a more accurate multi-language address conversion effect is achieved.

The invention relates to an intelligent real-time language synchronous translation system and a terminal thereof. The system comprises a voice segmentation module used for carrying out semantic analysis on target voice to be translated and segmenting the target voice according to obtained semantic information to obtain voice units; the scene recognition module is used for analyzing the voice unit to obtain scene information; the syntactic rhythm analysis module is used for converting the voice unit into a text, performing syntactic analysis, and analyzing the voice unit in combination with the information and the semantic information to obtain voice rhythm information and statement structure information; the semantic optimization module is used for correcting the semantic information based on the statement structure information, the scene information and the voice rhythm information to obtain optimized semantic information; and the speech translation module is used for translating the optimized semantic information based on the target language to obtain a target language translation. According to the system, through multi-module cooperation, voice semantics can be accurately mastered, different scenes are fitted, and the accuracy, smoothness and practicability of translation are effectively improved.

Systems and methods for providing one-to-one and audio and video calls or for providing multi-party audio or video conferences also provide language translation services. When language translation services are provided, a party to a call or conference hears both the audio of the speaker, and a translated version of the speaker's audio.

The present disclosure provides a real-time language translation system embodied in a physical device. Further, the real-time language translation system may include an input device which may be configured for receiving generating a user input data representing a linguistic input from a user. Further, the linguistic input corresponds to a user language. Further, the real-time language translation system may include a processing device which may be configured for generating a translation data based on the user input data. Further, the translation data represents a translation of the linguistic input. Further, the generating may be based on an AI module. Further, the processing device may be communicatively coupled to the input device. Further, the real-time language translation system may include a presentation device which may be configured for presenting the translation data. Further, the presentation device may be communicatively coupled to the processing device.

This disclosure provides a method, apparatus, electronic device, and storage medium for animal language conversion, relating to the field of artificial intelligence technology, specifically machine learning, deep learning, and natural language processing. The specific implementation involves: acquiring multimodal data related to the animal, including animal vocal data, animal behavioral data, and animal physical characteristics data; preprocessing the multimodal data to obtain fused multimodal data; identifying the animal's current emotion based on the fused multimodal data to obtain an emotion recognition result; and performing semantic mapping and language translation on the emotion recognition result to convert the animal language into human language, obtaining a language conversion result. This disclosure can accurately identify the animal's current emotional state and convert it into human language, thereby achieving deeper emotional communication and understanding between animals and humans, and improving the accuracy and efficiency of cross-species communication.

The invention relates to the technical field of language translation, and discloses a multi-language intelligent translation system based on artificial intelligence, and the system comprises an acquisition module which is used for receiving externally inputted voice information or text information; the corpus management module is used for receiving the text information transmitted by the acquisition module; the translation quality improving module is used for providing semantic mapping data based on the corpus management module; the privacy and security module is used for receiving the translation result generated by the translation quality improvement module; and the cloud-end cooperation module is used for coordinating task scheduling among the acquisition module, the corpus management module, the translation quality improvement module and the privacy and security module. Semantic mapping data of high resources and scarce languages is generated through a corpus management module and is called by a translation quality improvement module; the translation quality improving module generates target language output through context modeling and multi-stage translation, efficient cooperation of multi-language and scarce language translation can be achieved, and translation efficiency and accuracy are improved.

The invention discloses a sign language translation model, a sign language translation system and a sign language translation method based on full-modal alignment. The sign language translation method comprises the steps of extracting multi-modal features of hand, face and body postures from an input video and performing preliminary fusion; deep secondary fusion and alignment are carried out through multi-scale time sequence coding and a cross-modal collaborative attention mechanism, and a spatial-temporal feature sequence of full-modal alignment is generated; performing boundary detection and dynamic segmentation on the feature sequence by using a CTC-based sequence prediction model, and outputting a discrete sign language word sequence with a timestamp; and finally, capturing a sign language grammar structure of the sequence through a graph structure enhanced Transform encoder, inputting the sign language grammar structure into a Transform decoder integrating grammar consistency loss, and generating a target text conforming to target natural language grammar and semantic rules. According to the method, the problems of continuous sign language action adhesion and grammar structure difference are effectively solved, and the sign language translation accuracy and the natural language generation fluency are greatly improved.

Provided are computer-implemented systems and methods for providing improved rule-based machine translation, including a memory having a plurality of text translations, each text translation comprising an input text element and an output text element; at a processor: generating a text translation cluster comprising a plurality of matching text translations in the plurality of text translations, the text translation cluster associating the plurality of matching text translations; receiving the plurality of matching text translations associated with the text translation cluster; generating a first translation rule corresponding to the input text elements of the text translation cluster and a second translation rule corresponding to the output text elements; generating a machine translation rule based on the first translation rule and the second translation rule, the machine translation rule comprising a template and at least one converter; receiving an input text element for translation; and determining an output translated text element.

A method may include obtaining, at a system, audio during a communication session that includes a first device and a second device. In these and other embodiments, the audio may originate at the second device. The method may further include selecting, automatically and independently by the system based on one or more features of the communication session, a first translation process instead of a second translation process to translate between sign language and language data during the communication session. The method may also include using the selected one of the first translation process and the second translation process to generate a video that includes sign language content based on the audio.

The invention relates to a multi-language large model translation performance optimization method and system based on prompt strategy activation, and belongs to the field of natural language processing. In a low-resource language environment, the translation performance of a multi-language large model still has insufficiency. The understanding ability of a large model on low-resource language input semantics is limited, so that part of potential translation ability of the large model is not fully activated, in order to relieve the influence, prompts are screened from a high-quality corpus based on four prompt strategies, a context learning template is constructed to serve as large model user input, and therefore the user input of the large model is improved. Task guidance serves as system input, and finally performance optimization of a large model in a low-resource language translation task is achieved by dynamically adjusting a prompt strategy and model parameters. According to the method, the potential translation ability of a large model can be effectively activated, and the translation quality of the large model in a low-resource language environment is remarkably improved.

A computer-implemented system and computer instructions stored on non-transitory computer readable medium for bilingual medical inquiry in both Arabic and English, including multiple-choice question answering, open-ended question answering, and multi-turn question answering. The system and instructions use a mixture of experts large language model (MOE LLM) having a router network connected to multiple expert networks. The MOE LLM is trained with medical domain data and is used to receive the input bilingual text in a format for a medical inquiry, and output text in a format of a response to the medical inquiry, in sequence. The system and instructions incorporate an English-to-Arabic translation pipeline having a language translation model to generate Arabic language medical instruction sets from English language medical instructions, for large scale use in Arabic and English medical inquiry.

The invention discloses a high-precision geographical name translation method integrating artificial intelligence and multi-language syllable segmentation, and relates to the technical field of geographical name translations, the method comprises the following steps: firstly splitting an input geographical name into a sub-word Token sequence, and using a context coding model of a word embedding model to realize semantic context associated coding of a to-be-translated geographical name; and then modeling internal structured information and a dependency relationship in context semantics of the token to be translated in a local semantic relevance reconstruction enhancement mode, identifying recessive syllable relevance in compound words and adhesive words, and enhancing local morphological characteristics near syllable boundaries, so as to solve the problems of irregular spelling and cross-language interference, and improve the translation efficiency of the token to be translated. And further decoding and outputting the geographical name character string after syllable segmentation to carry out geographical name translation, generating a corresponding geographical name target language translation text, and realizing end-to-end effective conversion from an original geographical name to a target translation name.

Video relay services, communication systems, non-transitory machine-readable storage media, and methods are disclosed herein. A video relay service may include at least one server configured to receive a video stream including sign language content from a video communication device during a real-time communication session. The server may also be configured to automatically translate the sign language content into a verbal language translation during the real-time communication session without assistance of a human sign language interpreter. Further, the server may be configured to transmit the verbal language translation during the real-time communication session.

The invention discloses a multilingual work order processing method and device based on a dynamic LoRA network, computer equipment and a storage medium, and the method comprises the steps: receiving multilingual voice input, and converting the voice input into a source language text in real time; identifying a feature vector of the source language text; calculating the correlation with a plurality of LoRA networks to be selected according to the feature vectors; determining an activation probability of each to-be-selected LoRA network based on the correlation, and dynamically activating a target LoRA network through a pre-configured gating network according to the activation probability; fusing the target LoRA network into a basic large language model to generate a fusion model; and performing cross-language translation and work order content generation on the source language text by using the fusion model, and outputting a standard work order of a target language so as to distribute the work order to a corresponding processing module according to the work order content. By dynamically activating the proper LoRA network, the efficiency and accuracy of multilingual work order processing are improved.

The invention relates to a multi-national universal conference language translation method, device and equipment and a storage medium, and the translation method comprises the steps: continuously collecting all on-site voice information through a sensor, and carrying out the voice information preprocessing; identifying each sounding main body in the first voice information and corresponding first target voice information, temporarily storing the first voice information in a buffer queue, and marking a timestamp and the sounding main body of each target voice information in the first voice information, processing the second voice information and other newly-added voice information according to the same processing flow of the first voice information; and extracting the multi-dimensional acoustic features of the marked first target voice information, generating translation content through a pre-trained hybrid model according to a preset target translation language, and outputting the translation content to an application scene in a preset mode, thereby realizing the real-time performance and accuracy of multi-national conference language translation, improving the efficiency and effect of an international conference, and improving the user experience. And the communication and cooperation of the conference are promoted.

A healthcare translation apparatus, system, and method for two-way language translation in a healthcare environment is presented herein. The system includes a patient device in communication with a healthcare device, and a translation module configured to facilitate translation of inputs provided at the patient and healthcare devices. When a patient enters an input at the patient device, that input is sent to the translation module for real-time, translation into the language chosen by the healthcare provider. Similarly, when a health care provider enters an input through the healthcare device, that input is sent to the translation module for real-time, translation into the language chosen by the patient. Upon termination of the session, any and all data corresponding to the communication session is permanently deleted from the system and apparatus.

The invention provides a video dubbing language conversion method, a video dubbing language conversion system and related equipment. The method comprises the following steps: acquiring audio track data from a video to be converted; carrying out human voice extraction on the audio track data and classifying according to roles to obtain a single speaker audio of each role; performing voice-to-text conversion on the single speaker audio of each role to obtain an original language copywriting of each role; performing sound cloning on the single speaker audio of each role to obtain a timbre model of each role; performing target language translation on the original language copywriting of each role to obtain a translated copywriting of each role; based on the translation copywriting of each role and the tone model of each role, performing text-to-voice conversion to obtain a translation audio of each role; and performing replacement of each role translation audio on the audio track data in the to-be-converted video to obtain a dubbing conversion video. According to the technical scheme, language video dubbing conversion combined with the tone of the speaker is achieved, the video is more diversified, and the user requirements can be better met.

Video relay services, communication systems, non-transitory machine-readable storage media, and methods are disclosed herein. A video relay service may include at least one server configured to receive a video stream including sign language content from a video communication device during a real-time communication session. The server may also be configured to automatically translate the sign language content into a verbal language translation during the real-time communication session without assistance of a human sign language interpreter. Further, the server may be configured to transmit the verbal language translation during the real-time communication session.

Today neural networks are used to enable autonomous vehicles and improve the quality of speech recognition, real-time language translation, and online search optimizations. However, operation of the neural networks for these applications consumes energy. Quantization of parameters used by the neural networks reduces the amount of memory needed to store the parameters while also reducing the power consumed during operation of the neural network. Matrix operations performed by the neural networks require many multiplication calculations, so reducing the number of bits that are multiplied reduces the energy that is consumed. Quantizing smaller sets of the parameters using a shared scale factor improves accuracy compared with quantizing larger sets of the parameters. Accuracy of the calculations may be maintained by quantizing and scaling the parameters using fine-grained per-vector scale factors. A vector includes one or more elements within a single dimension of a multi-dimensional matrix.

The invention discloses a low-resource language translation model training method based on a big language, which comprises the following steps of: continuously pre-training a basic big language model by utilizing a multilingual text corpus, and storing a plurality of middle check points as candidate models; selecting a model with the optimal downstream translation task performance from the candidate models based on the performance of the verification set, and performing instruction supervision fine tuning by using the parallel instruction data set to obtain an intermediate model; and finally, training the intermediate model by using a preference optimization algorithm to obtain a target translation model by using a preference data set consisting of preferred translation and rejected translation. According to the method, through three-step progressive training, the problems of data sparsity, insufficient model capability, single training method and the like in low-resource language translation are effectively solved, and the translation quality is remarkably improved.

The invention discloses a sign language translation method and system based on deep learning, and relates to the field of deep learning, and the method comprises the steps: employing a lightweight deep learning model to recognize human body key points based on the left hand, the right hand, the face and the whole in an obtained sign language video, and obtaining key point information; constructing a multi-stream key point attention model, wherein the multi-stream key point attention model comprises four key point attention networks; obtaining a key point sequence according to the key point information, decoupling the key point sequence into a left hand flow, a right hand flow, a face flow and a whole body flow, performing correlation analysis on the left hand flow, the right hand flow, the face flow and the whole body flow through four key point attention networks, and determining key point features corresponding to the left hand flow, the right hand flow, the face flow and the whole body flow; pooling and fusing the key point features to generate sign language intermediate annotations; and inputting the sign language intermediate annotation into the trained mBART large language model to obtain a translated text, so that the sign language translation speed is effectively improved.