129 results about "Language speech" patented technology

Training a mixed language speech recognition model, and speech recognition performed by the model, can be enhanced. Model manager can control training and / or operation of the model. Mixed language data generation manager can generate an enhanced mixed language dataset that can enhance training of the model. Fine tuner can facilitate configuring hyperparameters of the model. Audio-based information and transcript that are representative of the dataset can be applied to the model to facilitate model training. FAL evaluator can determine fidelity, accuracy, and latency of performance of speech recognition on the audio-based information and transcript by the model. Based on such determination, mixed language data generation process and / or hyperparameters can be updated to enhance further training of the model to enhance fidelity, accuracy, and / or latency regarding performance of speech recognition by the model. Model manager can control one or more iterations of model training.

The invention relates to the technical field of voice processing, can be applied to business scenes such as financial science and technology, medical health and the like, and discloses a cross-language voice migration synthesis method, device, equipment and medium. And training an acoustic model by adopting a hierarchical self-adaptive fine tuning strategy, fusing the phoneme sequence for reasoning and the tone mark to generate a representation sequence, and finally synthesizing a target language speech signal. According to the method, a sharing and separation parallel cross-language modeling structure is constructed, the accuracy of phoneme and tone modeling in a low-resource language is effectively improved, the target language acoustic model has higher generalization ability and migration efficiency in combination with a multi-stage self-adaptive fine tuning and training data enhancement strategy, and the method is suitable for being applied to the field of multi-language modeling. And finally, synchronous improvement of the voice naturalness and the tone fidelity is realized.

The invention provides a voice recognition method and system for constructing a small language based on whispertoken, and relates to the technical field of natural language processing and voice recognition, and the method comprises the steps: extracting all tokens related to a target small language in a whispertoken, and forming an initial candidate set; matching and analyzing the tokens in the initial candidate set and the collected training text corpus of the target small language, and counting the occurrence frequency of the tokens in the corpus; and screening high-frequency tokens according to a frequency statistical result, and supplementing low-frequency tokens to construct a dynamic vocabulary. According to the method, the vocabulary quality is improved, the model training efficiency is optimized, the speech recognition accuracy is enhanced, the model generalization ability is improved, and the model construction process is simplified, so that an efficient, accurate and easy-to-implement solution is provided for the field of minority language speech recognition.

The invention relates to a speech recognition method, device and equipment based on an end-to-end cross-language large model. The method comprises the following steps: constructing a cross-language large model; obtaining a training data set formed by multiple groups of voice text label pairs, and inputting the training data set into the cross-language large model for end-to-end training until a trained cross-language large model is obtained; a to-be-recognized voice signal is input into the trained cross-language model, and filter feature and wav2vec2 feature extraction and splicing, text embedded feature representation, dimension alignment, splicing and decoding translation are carried out through a voice coding module, a text coding module, a feature alignment module and a decoding module in the model in sequence. And outputting and obtaining the speech recognition text of the target language. By adopting the method, the accuracy and robustness of speech recognition can be improved, and cross-language speech recognition is realized.

The invention relates to the technical field of simultaneous interpretation, and discloses a simultaneous interpretation data processing method and system based on a POE microphone array. The method comprises the following steps: synchronously acquiring multi-language original audio streams and meeting place environment noise spectrum features through a distributed microphone array powered by the Ethernet; after time domain framing is carried out on the audio stream, adaptive filtering is carried out by using a dynamic noise reduction weight coefficient to obtain a primary pure voice segment; dividing the multi-language speech endpoint detection model into independent speech units with language labels through a pre-trained multi-language speech endpoint detection model, and matching a corresponding acoustic model to generate a phoneme-level time alignment sequence; comparing and outputting a term replacement instruction stream in real time in combination with a simultaneous transfer term library, and generating an intermediate semantic representation vector after fusion; and the low-delay encoder converts the voice parameter sequence into a target language voice parameter sequence, and drives the waveform synthesizer to generate final simultaneous transmission audio. The method optimizes the whole process processing, gives consideration to the simultaneous transmission accuracy and real-time performance, and is suitable for a multilingual meeting place scene.

The invention relates to an emotion recognition method and device in film and television play dubbing, and the method comprises the steps: obtaining a film and television play audio of a source language and a to-be-dubbed text of a target language; the movie and television play audio is processed by using an emotion recognition model corresponding to the trained source language, and target emotion features are extracted; wherein the emotion recognition model is based on an audio pre-training sub-model and a feature extraction sub-model, and is obtained by training marked training data; the feature extraction sub-model is used for extracting audio features, and the audio pre-training sub-model is used for carrying out feature coding on the extracted audio features to generate emotional features; and performing voice synthesis on the to-be-dubbed text based on the target emotional characteristics by using a voice generation model corresponding to the trained target language, and generating dubbing of the target language corresponding to the movie and television play audio. Through the dubbing method and device, the original emotion is reserved in dubbing, cross-language voice seamless conversion is achieved, and the dubbing achieves the original taste and flavor effect.

The invention discloses a multi-language speech recognition and interaction method of a humanoid robot, particularly relates to the technical field of speech recognition and interaction, and effectively separates background noise and target speech signals through a multi-channel high-sensitivity microphone array in combination with an adaptive noise elimination algorithm, thereby improving the speech signal recognition accuracy in a noisy environment and improving the speech recognition efficiency. A speech speed fluctuation abnormal coefficient, an intonation deviation coefficient and a background noise influence coefficient are calculated through deep analysis of speech signals and background noise feature data of a target interaction object, and a sentence segmentation model is adjusted according to habits of speech speeds, intonation, pauses and the like of different speakers, so that the problem of inaccurate sentence segmentation under different users or contexts is avoided; the sentence segmentation adaptation optimization index is generated according to the sentence segmentation adaptation optimization model, whether parameters of the sentence segmentation model need to be adjusted or not is judged by combining a threshold value, real-time response and optimization are achieved, the situation of misjudgment or missed judgment is reduced, and the smoothness and accuracy of overall interaction are improved.

The invention provides a voice recognition method, device and system, electronic equipment and a storage medium, and the method is applied to terminal equipment, and comprises the steps: carrying out the acoustic feature extraction of a voice signal based on the language information of the voice signal, obtaining an acoustic feature, decoding the acoustic feature, and obtaining a plurality of initial recognition results of the voice signal; determining a voice recognition result of the voice signal; the speech recognition result is obtained by applying a speech recognition model to perform semantic error correction on the basis of each initial recognition result, the acoustic feature and the language information; a speech recognition model is constructed on the basis of a large-scale language model, the defects that current multi-language speech recognition is low in accuracy and prone to misjudgment are overcome, through a two-step recognition process, a plurality of initial recognition results are generated locally and rapidly, and then the speech recognition accuracy is improved through the strong semantic understanding ability of the large-scale model. And the acoustic features and the language information are fused to carry out multi-modal deep error correction, so that the accuracy of multi-language speech recognition is greatly improved.

The invention discloses an artificial intelligence-assisted cross-language automatic note taking and term tagging system, and belongs to the technical field of language tagging. The system firstly obtains and processes audio data, generates a preliminary text through cross-language speech recognition, recognizes term candidate words in the text, and performs pronunciation association matching with a term knowledge base; and performing language recognition and context analysis on the candidate words and the contexts thereof, performing comprehensive pronunciation matching, language recognition and context score calculation to obtain a comprehensive score, thereby accurately selecting a cross-language term which most conforms to the current context as the final candidate word, and obtaining the annotation information of the final candidate word. And integrating the final candidate word and the annotation information of the final candidate word into the calibration note text, and outputting an automatic note with cross-language term annotations. According to the method, complex audios including language mixing and the like can be automatically processed, the high-quality calibration note text is generated, and the information recording efficiency and accuracy in a cross-language communication scene are remarkably improved.

The invention belongs to the field of artificial intelligence, is applied to the field of intelligent medical treatment, and relates to a cross-language speech generation method based on artificial intelligence and related equipment thereof.The cross-language speech generation method comprises the steps that received text information and language information are input into a text coding model, and semantic features are obtained; inputting the received audio information into an emotion capture model and a tone extraction model to obtain an emotion feature and a tone feature, and generating an emotion feature according to the emotion feature and the tone feature; inputting the semantic features into a pronunciation duration prediction model to obtain the duration of each pronunciation unit, and marking the corresponding semantic features through the durations of the pronunciation units to obtain marked features; inputting the mark features into a stream model to obtain voice data features; and inputting the voice data features and the emotion features into an audio decoding model to obtain a voice signal. The invention also relates to a block chain technology, and the voice signal can be stored in a block chain. According to the method and the device, the problem that synthesized cross-language voice is unnatural in expression and has a mechanical feeling is solved.

The invention discloses a multi-language speech recognition method. A multi-language speech recognition model comprises an encoder, a language classification model, a CTC module and a decoder. The method comprises the following steps: acquiring acoustic features of an audio to be identified, and inputting the acoustic features to the encoder to obtain an encoder output result containing acoustic feature local information and acoustic feature global information; a language classification model carries out preset processing on the encoder output result to obtain a language classification result; the CTC module is combined with a weighted finite state conversion machine, and an encoder output result is mapped to a streaming nbest identification result through a prefix beam search algorithm; and the decoder re-scores the output result of the encoder according to the streaming nbest identification result. The multi-language speech recognition model only shares an encoder at the bottom layer, and the language classification model, the CTC module and the decoder at the top layer are independent. The problem that homophones of different languages are confused in speech recognition is reduced through shared bottom layer parameters and a re-scoring process, and the language classification accuracy is improved.

The invention relates to enhancing voice-to-text performance of mixed languages. According to the embodiment of the invention, the mixed language speech recognition model is trained, so that speech recognition executed by the model can be enhanced. The model manager may control training and / or operation of the model. A hybrid language data generation manager may generate an enhanced hybrid language dataset, which may enhance training of the model. The trimmer can facilitate configuration of hyper-parameters of the model. Audio-based information and transcriptions representative of the dataset may be applied to the model to facilitate model training. The FAL evaluator may determine the fidelity, accuracy, and latency of the model to perform speech recognition on audio-based information and transcriptions. Based on this determination, the hybrid language data generation process and / or hyper-parameters may be updated to enhance further training of the model, thereby enhancing fidelity, accuracy, and / or latency with respect to speech recognition performance of the model. The model manager may control one or more iterations of the model training.

The invention discloses a personalized sound cloning method and device based on dynamic adaptation, equipment and a medium, and relates to the technical field of speech synthesis, and the method comprises the steps: carrying out the feature extraction of an original speech sample based on a preset deep neural network, so as to obtain a multi-modal speech feature, carrying out the fusion coding, and generating an acoustic representation model; determining to-be-synthesized text content and a target scene template, and extracting target acoustic parameters from a preset scene template library, so as to carry out acoustic parameter adjustment on the to-be-synthesized text content based on the target acoustic parameters and an acoustic representation model, and generating target cloned voice; extracting acoustic features irrelevant to languages by using a multi-language speech encoder, determining a to-be-migrated target language, and generating a corresponding text corpus; and carrying out phoneme mapping and alignment operation on the target clone voice based on the text corpus, and carrying out cross-language migration operation on the target clone voice through a preset generative adversarial network and the acoustic features to generate target language voice.

The invention discloses a cross-language voice interaction method and system based on multi-modal semantic understanding and a storage medium, and relates to the technical field of artificial intelligence and natural language processing. The method comprises the following steps: acquiring a source language speech stream, performing parallel double-flow feature extraction, extracting text semantic features through a mixed language code recognition model based on a unified phoneme space, and extracting acoustic features containing rhythm information at the same time; performing alignment fusion on the text and the acoustic features by using a cross-modal attention mechanism to generate multi-modal semantic representation; analyzing the explicit intention and the implicit emotion based on the representation, and generating a reply strategy and an emotion control label of the target language; and finally, synthesizing a target voice with a corresponding emotion color. According to the invention, the bottleneck that the traditional cascade architecture loses side language information is broken through, the precise understanding and strategic feedback of complex contexts such as Chinese-English mixed language codes (Code-mixing), anti-quiescence, hesitation and the like are realized, and the method is particularly suitable for transnational business negotiation and international customer service scenes.

The present invention discloses a method, device, storage medium, and electronic device for generating virtual speech. The method comprises: obtaining multiple different speech text samples and speech attribute information, wherein each speech text sample in multiple different language speech text samples corresponds to a language and an object; inputting each speech text sample into a multi-stream encoder to obtain text features corresponding to each speech text sample; and training a preset speech acoustic model based on generative adversarial network modeling using the text features and speech features to obtain a target acoustic model for generating virtual speech. The present invention can support cross-language data training and the generation of cross-language speakers. The multi-stream encoder can better capture text features in different languages, improve the flexibility and reliability of virtual preset generation, and thus solve the technical problem of low flexibility and reliability in generating virtual speech in the prior art.

The invention discloses a cross-language speech text retrieval method based on a pre-trained automatic speech recognition model. According to the method, a pre-trained automatic speech recognition model is expanded to a speech text retrieval system, and the model is finely adjusted in combination with comparative learning and a low-rank adaptive method, so that an efficient cross-language speech text retrieval function is realized. The method is based on an encoder-decoder structure initialized by a pre-training model, and comprises the following steps: firstly, converting voice data and text data into high-dimensional feature vectors through an encoding module respectively, and mapping the high-dimensional feature vectors to a unified embedding space; the model then minimizes the matching speech and text embedding distance in the embedding space. And finally, through a similarity matching algorithm, the model can efficiently match the query voice with the text in the text library, thereby returning the most relevant text data. Experimental results show that the retrieval precision and efficiency of the method on a test data set are close to or superior to those of an existing public model, and it is proved that the method has wide application prospects and remarkable practical value.

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a multi-language speech recognition method, which comprises the following steps: acquiring a pre-trained basic speech recognition model; an adapter is correspondingly configured for each language; configuring a plurality of language characteristic expert modules; acquiring training voice data, selecting a corresponding language characteristic expert module and inputting a voice sample; and determining a target low-resource language, freezing parameters of other adapters, only training the adapter of the target language, selecting the corresponding adapter based on the language identifier and performing training until training of all batches of data is completed, and generating a target speech recognition model. According to the method, the language dependence adapter and the language characteristic expert module are configured, so that the characteristics of different languages are processed in a targeted manner, and the adaptability and the recognition accuracy of the model are remarkably improved. Through freezing the adapter of the non-target language, unnecessary resource consumption is reduced, optimization of the target low-resource language is focused, and high efficiency of the training process is ensured.

The invention relates to the technical field of natural language processing, and discloses a Yi language speech recognition method based on self-supervision and attention feature fusion. The method comprises a feature encoder module, a comparative learning module, a mask language modeling module, a joint optimization and feature fusion module and a decoder module, the feature encoder module adopts a convolutional neural network structure and converts continuous waveform signals into feature representation suitable for subsequent modeling, and the comparative learning module performs feature fusion on the continuous waveform signals through a Gumbel-Softmax technology. The method comprises the following steps that: a mask language modeling module and a feature fusion module are integrated, deviation caused by manual definition or clustering is avoided, the mask language modeling module obviously enhances semantic understanding and tone modeling capabilities of a model in a low-resource scene, a self-attention feature fusion mechanism is introduced into the feature fusion module, continuous features, discrete unit representation and semantic context representation from an acoustic level are spliced, and a self-attention feature fusion mechanism is introduced into the self-attention feature fusion mechanism. The decoder module adopts a decoder structure based on connection time sequence classification, and the corresponding relation between the voice and the text can be achieved without strict alignment labeling.

A method for training a speech translation model includes: obtaining a trained first text translation model and a speech recognition model, and constructing a candidate speech translation model to be trained based on the first text translation model and the speech recognition model; obtaining at least one of a first sample source language speech or a first sample source language text to obtain a training sample of the candidate speech translation model; and training the candidate speech translation model based on the training sample until the training is completed, and obtaining a trained target speech translation model.

The invention provides a speech translation method and device, and the method comprises the steps: translating source language speech data based on a speech translation model, and obtaining a target language text; a training target of the speech translation model comprises minimizing a difference between a first target language prediction text generated based on source language sample speech data and a translation label corresponding to the source language sample speech data, and minimizing a difference between speech features of the source language sample speech data and text features of a first source language sample text. And minimizing the difference between a second target language prediction text generated based on the pseudo source language speech features and a translation label corresponding to the second source language sample text. According to the method and the device, the problem of scarcity of annotated voice data is solved by efficiently utilizing relatively rich text data in a low-resource language scene, so that the performance of a voice translation model is improved.

The invention discloses a cross-language speech recognition method, system and device and a storage medium, and the method comprises the steps: carrying out the preprocessing of training speech, and obtaining a training speech frame sequence; extracting content characterization, speaker characterization and pitch characterization from the training voice frame sequence; performing voice reconstruction according to the content representation, the speaker representation and the pitch representation to obtain a target language voice; based on the training voice, constructing a target language voice recognition model according to the target language voice; in response to the target language recognition instruction, obtaining a target voice; and inputting the target voice into the target language voice recognition model to obtain a recognition result output by the target language voice recognition model, so that accurate modeling of dialect acoustic characteristics can be realized under the condition of extremely low annotation data through cross-language feature decoupling and a self-supervised migration mechanism. The recognition robustness of the complex tones and the characteristic vocabularies of the Guiwili is remarkably improved, and efficient generalization application in a dialect scene is achieved.

A system for translating speech from at least two source languages into another target language provides direct speech to target language translation. The target text is converted to speech in the target language through a TTS system. The system simplifies speech recognition and translation process by providing direct translation, includes mechanisms described herein that facilitate mixed language source speech translation, and punctuating output text streams in the target language. It also in some embodiments allows translation of speech into the target language to reflect the voice of the speaker of the source speech based on characteristics of the source language speech and speaker's voice and to produce subtitled data in the target language corresponding to the source speech. The system uses models having been trained using (i) encoder-decoder architectures with attention mechanisms and training data using TTS and (ii) parallel text training data in more than two different languages.

Techniques for performing tasks are provided. An example method includes receiving, via the one or more input devices, a natural-language speech input including a request to perform a task; providing, at a language model, a plan corresponding to the task; determining whether the plan satisfies a set of resolution criteria; in accordance with a determination that the plan satisfies the set of resolution criteria, initiating performance of the task according to the selected plan; and in accordance with a determination that the plan does not satisfy the set of resolution criteria: providing a query to an information retrieval service requesting a set of resolution data; receiving, from the information retrieval service, the set of resolution data; resolving the plan based on the set of resolution data; and initiating performance of the task according to the resolved plan.

The invention discloses a Tibetan language Andole speech enhancement method suitable for specific noise and general noise, and relates to the technical field of low-resource language speech processing. The method specifically comprises the following steps: acquiring noisy voice information; preprocessing the noisy voice information, and dividing the processed data into a training set and a test set; a noise spectrum feature input by the noise classification module is optimized based on a weighted average method of a logarithmic power spectrum domain hard threshold; constructing a noise classification task and noise suppression task fused multi-target Tibetan speech enhancement model; training and testing the built speech enhancement model by using the training set and the test set to obtain a trained speech enhancement model; and inputting the actually collected noisy speech into the trained speech enhancement model to obtain an enhanced logarithmic power spectrum. Through effective fusion of the noise classification task and the suppression task, the quality and the intelligibility of the Tibetan language Andor dialect enhancement voice are improved, the human-computer interaction of the Tibetan language Andor dialect is more perfect, and the method has a positive effect on popularization of Tibetan language informatization.

The application provides a speech recognition method and system for constructing a small language based on a whisper token, and relates to the technical field of natural language processing and speech recognition. The method comprises the following steps: extracting all tokens related to a target small language in a whisper tokenizer to form an initial candidate set; matching and analyzing the tokens in the initial candidate set with collected training text corpus of the target small language, and counting the frequency of the tokens in the corpus; and screening high-frequency tokens and supplementing low-frequency tokens according to the frequency counting result to construct a dynamic vocabulary. The application improves the vocabulary quality, optimizes the model training efficiency, enhances the speech recognition accuracy, improves the model generalization ability, and simplifies the model construction process, thereby providing an efficient, accurate and easy-to-implement solution for the field of small language speech recognition.

The present invention provides a method, apparatus, electronic device and storage medium for translation quality assessment. The method includes: determining a source language speech and a target language text; performing source language standard translation on the source language speech to obtain a source language standard text, and determining a speech assessment result based on the source language standard text; determining a text assessment result based on the target language text; and determining the translation quality of the target language text based on the speech assessment result and the text assessment result. The translation quality assessment method, apparatus, electronic device and storage medium provided by the embodiments of the present invention take into account both the quality of the source language speech and the quality of the target language text, thereby improving the accuracy of translation quality assessment.

Methods and systems for transcribing communications are provided. Methods may include receiving a communication. Methods may include splitting the communication into a plurality of communication segments. Each communication segment may include two or more words. Methods may include transcribing each segment included in the plurality of communication segments, in parallel. The transcribing may include using a transformer neural network to transcribe each segment included in the plurality of communication segments. Methods may include generating a transcription from the transcribing. The transcription may be generated by combining the transcription of each of the communication segments into a combined transcription. Methods may include correcting the combined transcription.

The application relates to a speech recognition method and a server. The method comprises the following steps: obtaining a to-be-recognized speech signal; recognizing each frame of the to-be-recognized speech signal according to an acoustic model of each language, and respectively outputting corresponding language phonemes and prediction probabilities; wherein the acoustic model of each language is respectively constructed according to shared hidden layer training; sequentially traversing a sentence decoding graph and a multi-language slot decoding graph connected with each other to obtain a corresponding path; wherein the sentence decoding graph is used for decoding phonemes entering a non-slot, and the slot decoding graph is used for decoding phonemes entering a slot; when it is determined that the path passes through the multi-language slot decoding graph in the speech decoding graph, screening the path according to the prediction probabilities of the language phonemes corresponding to each language, and determining the text information corresponding to the target path as a speech recognition result. The scheme provided by the application can accurately recognize mixed multi-language speech information.

The application discloses a speech synthesis method and device, electronic equipment and a computer readable medium, and relates to the technical field of speech synthesis. The method comprises the following steps: based on input text, obtaining a first synthesized speech according to a pre-acquired basic language speech synthesis model, and obtaining a second synthesized speech according to a pre-acquired target language speech synthesis model, wherein the similarity of the training speech of the target language speech synthesis model to the training speech of the basic language speech synthesis model is higher than a preset value; performing speech conversion on the second synthesized speech based on pre-acquired basic language training speech to obtain third synthesized speech; and obtaining target synthesized speech based on the first synthesized speech and the third synthesized speech. Therefore, the similarity of synthesized speech of different languages is further improved, and the target synthesized speech including bilingual or even multilingual speech has high timbre consistency, thereby improving the hearing effect.

A computer-implemented method, computer program product and computing system for: receiving speech in a source language to define source language speech; performing a first token-based transcription of the source language speech into text of the source language using a first look-ahead encoder to define source language text; and performing a first token-based translation of the source language speech into text of a target language using a second look-ahead encoder to define target language text, wherein the first look-ahead encoder is smaller than the second look-ahead encoder.