115 results about "Language speech" patented technology

Training a mixed language speech recognition model, and speech recognition performed by the model, can be enhanced. Model manager can control training and / or operation of the model. Mixed language data generation manager can generate an enhanced mixed language dataset that can enhance training of the model. Fine tuner can facilitate configuring hyperparameters of the model. Audio-based information and transcript that are representative of the dataset can be applied to the model to facilitate model training. FAL evaluator can determine fidelity, accuracy, and latency of performance of speech recognition on the audio-based information and transcript by the model. Based on such determination, mixed language data generation process and / or hyperparameters can be updated to enhance further training of the model to enhance fidelity, accuracy, and / or latency regarding performance of speech recognition by the model. Model manager can control one or more iterations of model training.

The invention relates to the technical field of voice processing, can be applied to business scenes such as financial science and technology, medical health and the like, and discloses a cross-language voice migration synthesis method, device, equipment and medium. And training an acoustic model by adopting a hierarchical self-adaptive fine tuning strategy, fusing the phoneme sequence for reasoning and the tone mark to generate a representation sequence, and finally synthesizing a target language speech signal. According to the method, a sharing and separation parallel cross-language modeling structure is constructed, the accuracy of phoneme and tone modeling in a low-resource language is effectively improved, the target language acoustic model has higher generalization ability and migration efficiency in combination with a multi-stage self-adaptive fine tuning and training data enhancement strategy, and the method is suitable for being applied to the field of multi-language modeling. And finally, synchronous improvement of the voice naturalness and the tone fidelity is realized.

The invention provides a voice recognition method and system for constructing a small language based on whispertoken, and relates to the technical field of natural language processing and voice recognition, and the method comprises the steps: extracting all tokens related to a target small language in a whispertoken, and forming an initial candidate set; matching and analyzing the tokens in the initial candidate set and the collected training text corpus of the target small language, and counting the occurrence frequency of the tokens in the corpus; and screening high-frequency tokens according to a frequency statistical result, and supplementing low-frequency tokens to construct a dynamic vocabulary. According to the method, the vocabulary quality is improved, the model training efficiency is optimized, the speech recognition accuracy is enhanced, the model generalization ability is improved, and the model construction process is simplified, so that an efficient, accurate and easy-to-implement solution is provided for the field of minority language speech recognition.

The invention relates to the technical field of simultaneous interpretation, and discloses a simultaneous interpretation data processing method and system based on a POE microphone array. The method comprises the following steps: synchronously acquiring multi-language original audio streams and meeting place environment noise spectrum features through a distributed microphone array powered by the Ethernet; after time domain framing is carried out on the audio stream, adaptive filtering is carried out by using a dynamic noise reduction weight coefficient to obtain a primary pure voice segment; dividing the multi-language speech endpoint detection model into independent speech units with language labels through a pre-trained multi-language speech endpoint detection model, and matching a corresponding acoustic model to generate a phoneme-level time alignment sequence; comparing and outputting a term replacement instruction stream in real time in combination with a simultaneous transfer term library, and generating an intermediate semantic representation vector after fusion; and the low-delay encoder converts the voice parameter sequence into a target language voice parameter sequence, and drives the waveform synthesizer to generate final simultaneous transmission audio. The method optimizes the whole process processing, gives consideration to the simultaneous transmission accuracy and real-time performance, and is suitable for a multilingual meeting place scene.

The invention relates to an emotion recognition method and device in film and television play dubbing, and the method comprises the steps: obtaining a film and television play audio of a source language and a to-be-dubbed text of a target language; the movie and television play audio is processed by using an emotion recognition model corresponding to the trained source language, and target emotion features are extracted; wherein the emotion recognition model is based on an audio pre-training sub-model and a feature extraction sub-model, and is obtained by training marked training data; the feature extraction sub-model is used for extracting audio features, and the audio pre-training sub-model is used for carrying out feature coding on the extracted audio features to generate emotional features; and performing voice synthesis on the to-be-dubbed text based on the target emotional characteristics by using a voice generation model corresponding to the trained target language, and generating dubbing of the target language corresponding to the movie and television play audio. Through the dubbing method and device, the original emotion is reserved in dubbing, cross-language voice seamless conversion is achieved, and the dubbing achieves the original taste and flavor effect.

The invention provides a voice recognition method, device and system, electronic equipment and a storage medium, and the method is applied to terminal equipment, and comprises the steps: carrying out the acoustic feature extraction of a voice signal based on the language information of the voice signal, obtaining an acoustic feature, decoding the acoustic feature, and obtaining a plurality of initial recognition results of the voice signal; determining a voice recognition result of the voice signal; the speech recognition result is obtained by applying a speech recognition model to perform semantic error correction on the basis of each initial recognition result, the acoustic feature and the language information; a speech recognition model is constructed on the basis of a large-scale language model, the defects that current multi-language speech recognition is low in accuracy and prone to misjudgment are overcome, through a two-step recognition process, a plurality of initial recognition results are generated locally and rapidly, and then the speech recognition accuracy is improved through the strong semantic understanding ability of the large-scale model. And the acoustic features and the language information are fused to carry out multi-modal deep error correction, so that the accuracy of multi-language speech recognition is greatly improved.

The invention discloses an artificial intelligence-assisted cross-language automatic note taking and term tagging system, and belongs to the technical field of language tagging. The system firstly obtains and processes audio data, generates a preliminary text through cross-language speech recognition, recognizes term candidate words in the text, and performs pronunciation association matching with a term knowledge base; and performing language recognition and context analysis on the candidate words and the contexts thereof, performing comprehensive pronunciation matching, language recognition and context score calculation to obtain a comprehensive score, thereby accurately selecting a cross-language term which most conforms to the current context as the final candidate word, and obtaining the annotation information of the final candidate word. And integrating the final candidate word and the annotation information of the final candidate word into the calibration note text, and outputting an automatic note with cross-language term annotations. According to the method, complex audios including language mixing and the like can be automatically processed, the high-quality calibration note text is generated, and the information recording efficiency and accuracy in a cross-language communication scene are remarkably improved.

The invention relates to enhancing voice-to-text performance of mixed languages. According to the embodiment of the invention, the mixed language speech recognition model is trained, so that speech recognition executed by the model can be enhanced. The model manager may control training and / or operation of the model. A hybrid language data generation manager may generate an enhanced hybrid language dataset, which may enhance training of the model. The trimmer can facilitate configuration of hyper-parameters of the model. Audio-based information and transcriptions representative of the dataset may be applied to the model to facilitate model training. The FAL evaluator may determine the fidelity, accuracy, and latency of the model to perform speech recognition on audio-based information and transcriptions. Based on this determination, the hybrid language data generation process and / or hyper-parameters may be updated to enhance further training of the model, thereby enhancing fidelity, accuracy, and / or latency with respect to speech recognition performance of the model. The model manager may control one or more iterations of the model training.

The invention discloses a personalized sound cloning method and device based on dynamic adaptation, equipment and a medium, and relates to the technical field of speech synthesis, and the method comprises the steps: carrying out the feature extraction of an original speech sample based on a preset deep neural network, so as to obtain a multi-modal speech feature, carrying out the fusion coding, and generating an acoustic representation model; determining to-be-synthesized text content and a target scene template, and extracting target acoustic parameters from a preset scene template library, so as to carry out acoustic parameter adjustment on the to-be-synthesized text content based on the target acoustic parameters and an acoustic representation model, and generating target cloned voice; extracting acoustic features irrelevant to languages by using a multi-language speech encoder, determining a to-be-migrated target language, and generating a corresponding text corpus; and carrying out phoneme mapping and alignment operation on the target clone voice based on the text corpus, and carrying out cross-language migration operation on the target clone voice through a preset generative adversarial network and the acoustic features to generate target language voice.

The invention discloses a cross-language voice interaction method and system based on multi-modal semantic understanding and a storage medium, and relates to the technical field of artificial intelligence and natural language processing. The method comprises the following steps: acquiring a source language speech stream, performing parallel double-flow feature extraction, extracting text semantic features through a mixed language code recognition model based on a unified phoneme space, and extracting acoustic features containing rhythm information at the same time; performing alignment fusion on the text and the acoustic features by using a cross-modal attention mechanism to generate multi-modal semantic representation; analyzing the explicit intention and the implicit emotion based on the representation, and generating a reply strategy and an emotion control label of the target language; and finally, synthesizing a target voice with a corresponding emotion color. According to the invention, the bottleneck that the traditional cascade architecture loses side language information is broken through, the precise understanding and strategic feedback of complex contexts such as Chinese-English mixed language codes (Code-mixing), anti-quiescence, hesitation and the like are realized, and the method is particularly suitable for transnational business negotiation and international customer service scenes.

The present invention discloses a method, device, storage medium, and electronic device for generating virtual speech. The method comprises: obtaining multiple different speech text samples and speech attribute information, wherein each speech text sample in multiple different language speech text samples corresponds to a language and an object; inputting each speech text sample into a multi-stream encoder to obtain text features corresponding to each speech text sample; and training a preset speech acoustic model based on generative adversarial network modeling using the text features and speech features to obtain a target acoustic model for generating virtual speech. The present invention can support cross-language data training and the generation of cross-language speakers. The multi-stream encoder can better capture text features in different languages, improve the flexibility and reliability of virtual preset generation, and thus solve the technical problem of low flexibility and reliability in generating virtual speech in the prior art.

The invention discloses a cross-language speech text retrieval method based on a pre-trained automatic speech recognition model. According to the method, a pre-trained automatic speech recognition model is expanded to a speech text retrieval system, and the model is finely adjusted in combination with comparative learning and a low-rank adaptive method, so that an efficient cross-language speech text retrieval function is realized. The method is based on an encoder-decoder structure initialized by a pre-training model, and comprises the following steps: firstly, converting voice data and text data into high-dimensional feature vectors through an encoding module respectively, and mapping the high-dimensional feature vectors to a unified embedding space; the model then minimizes the matching speech and text embedding distance in the embedding space. And finally, through a similarity matching algorithm, the model can efficiently match the query voice with the text in the text library, thereby returning the most relevant text data. Experimental results show that the retrieval precision and efficiency of the method on a test data set are close to or superior to those of an existing public model, and it is proved that the method has wide application prospects and remarkable practical value.

The invention relates to the technical field of natural language processing, and discloses a Yi language speech recognition method based on self-supervision and attention feature fusion. The method comprises a feature encoder module, a comparative learning module, a mask language modeling module, a joint optimization and feature fusion module and a decoder module, the feature encoder module adopts a convolutional neural network structure and converts continuous waveform signals into feature representation suitable for subsequent modeling, and the comparative learning module performs feature fusion on the continuous waveform signals through a Gumbel-Softmax technology. The method comprises the following steps that: a mask language modeling module and a feature fusion module are integrated, deviation caused by manual definition or clustering is avoided, the mask language modeling module obviously enhances semantic understanding and tone modeling capabilities of a model in a low-resource scene, a self-attention feature fusion mechanism is introduced into the feature fusion module, continuous features, discrete unit representation and semantic context representation from an acoustic level are spliced, and a self-attention feature fusion mechanism is introduced into the self-attention feature fusion mechanism. The decoder module adopts a decoder structure based on connection time sequence classification, and the corresponding relation between the voice and the text can be achieved without strict alignment labeling.

The invention provides a speech translation method and device, and the method comprises the steps: translating source language speech data based on a speech translation model, and obtaining a target language text; a training target of the speech translation model comprises minimizing a difference between a first target language prediction text generated based on source language sample speech data and a translation label corresponding to the source language sample speech data, and minimizing a difference between speech features of the source language sample speech data and text features of a first source language sample text. And minimizing the difference between a second target language prediction text generated based on the pseudo source language speech features and a translation label corresponding to the second source language sample text. According to the method and the device, the problem of scarcity of annotated voice data is solved by efficiently utilizing relatively rich text data in a low-resource language scene, so that the performance of a voice translation model is improved.

The invention discloses a cross-language speech recognition method, system and device and a storage medium, and the method comprises the steps: carrying out the preprocessing of training speech, and obtaining a training speech frame sequence; extracting content characterization, speaker characterization and pitch characterization from the training voice frame sequence; performing voice reconstruction according to the content representation, the speaker representation and the pitch representation to obtain a target language voice; based on the training voice, constructing a target language voice recognition model according to the target language voice; in response to the target language recognition instruction, obtaining a target voice; and inputting the target voice into the target language voice recognition model to obtain a recognition result output by the target language voice recognition model, so that accurate modeling of dialect acoustic characteristics can be realized under the condition of extremely low annotation data through cross-language feature decoupling and a self-supervised migration mechanism. The recognition robustness of the complex tones and the characteristic vocabularies of the Guiwili is remarkably improved, and efficient generalization application in a dialect scene is achieved.

A system for translating speech from at least two source languages into another target language provides direct speech to target language translation. The target text is converted to speech in the target language through a TTS system. The system simplifies speech recognition and translation process by providing direct translation, includes mechanisms described herein that facilitate mixed language source speech translation, and punctuating output text streams in the target language. It also in some embodiments allows translation of speech into the target language to reflect the voice of the speaker of the source speech based on characteristics of the source language speech and speaker's voice and to produce subtitled data in the target language corresponding to the source speech. The system uses models having been trained using (i) encoder-decoder architectures with attention mechanisms and training data using TTS and (ii) parallel text training data in more than two different languages.

Techniques for performing tasks are provided. An example method includes receiving, via the one or more input devices, a natural-language speech input including a request to perform a task; providing, at a language model, a plan corresponding to the task; determining whether the plan satisfies a set of resolution criteria; in accordance with a determination that the plan satisfies the set of resolution criteria, initiating performance of the task according to the selected plan; and in accordance with a determination that the plan does not satisfy the set of resolution criteria: providing a query to an information retrieval service requesting a set of resolution data; receiving, from the information retrieval service, the set of resolution data; resolving the plan based on the set of resolution data; and initiating performance of the task according to the resolved plan.

The application provides a speech recognition method and system for constructing a small language based on a whisper token, and relates to the technical field of natural language processing and speech recognition. The method comprises the following steps: extracting all tokens related to a target small language in a whisper tokenizer to form an initial candidate set; matching and analyzing the tokens in the initial candidate set with collected training text corpus of the target small language, and counting the frequency of the tokens in the corpus; and screening high-frequency tokens and supplementing low-frequency tokens according to the frequency counting result to construct a dynamic vocabulary. The application improves the vocabulary quality, optimizes the model training efficiency, enhances the speech recognition accuracy, improves the model generalization ability, and simplifies the model construction process, thereby providing an efficient, accurate and easy-to-implement solution for the field of small language speech recognition.

The present invention provides a method, apparatus, electronic device and storage medium for translation quality assessment. The method includes: determining a source language speech and a target language text; performing source language standard translation on the source language speech to obtain a source language standard text, and determining a speech assessment result based on the source language standard text; determining a text assessment result based on the target language text; and determining the translation quality of the target language text based on the speech assessment result and the text assessment result. The translation quality assessment method, apparatus, electronic device and storage medium provided by the embodiments of the present invention take into account both the quality of the source language speech and the quality of the target language text, thereby improving the accuracy of translation quality assessment.

Methods and systems for transcribing communications are provided. Methods may include receiving a communication. Methods may include splitting the communication into a plurality of communication segments. Each communication segment may include two or more words. Methods may include transcribing each segment included in the plurality of communication segments, in parallel. The transcribing may include using a transformer neural network to transcribe each segment included in the plurality of communication segments. Methods may include generating a transcription from the transcribing. The transcription may be generated by combining the transcription of each of the communication segments into a combined transcription. Methods may include correcting the combined transcription.

The application relates to a speech recognition method and a server. The method comprises the following steps: obtaining a to-be-recognized speech signal; recognizing each frame of the to-be-recognized speech signal according to an acoustic model of each language, and respectively outputting corresponding language phonemes and prediction probabilities; wherein the acoustic model of each language is respectively constructed according to shared hidden layer training; sequentially traversing a sentence decoding graph and a multi-language slot decoding graph connected with each other to obtain a corresponding path; wherein the sentence decoding graph is used for decoding phonemes entering a non-slot, and the slot decoding graph is used for decoding phonemes entering a slot; when it is determined that the path passes through the multi-language slot decoding graph in the speech decoding graph, screening the path according to the prediction probabilities of the language phonemes corresponding to each language, and determining the text information corresponding to the target path as a speech recognition result. The scheme provided by the application can accurately recognize mixed multi-language speech information.

The application discloses a speech synthesis method and device, electronic equipment and a computer readable medium, and relates to the technical field of speech synthesis. The method comprises the following steps: based on input text, obtaining a first synthesized speech according to a pre-acquired basic language speech synthesis model, and obtaining a second synthesized speech according to a pre-acquired target language speech synthesis model, wherein the similarity of the training speech of the target language speech synthesis model to the training speech of the basic language speech synthesis model is higher than a preset value; performing speech conversion on the second synthesized speech based on pre-acquired basic language training speech to obtain third synthesized speech; and obtaining target synthesized speech based on the first synthesized speech and the third synthesized speech. Therefore, the similarity of synthesized speech of different languages is further improved, and the target synthesized speech including bilingual or even multilingual speech has high timbre consistency, thereby improving the hearing effect.

A computer-implemented method, computer program product and computing system for: receiving speech in a source language to define source language speech; performing a first token-based transcription of the source language speech into text of the source language using a first look-ahead encoder to define source language text; and performing a first token-based translation of the source language speech into text of a target language using a second look-ahead encoder to define target language text, wherein the first look-ahead encoder is smaller than the second look-ahead encoder.

The invention discloses a vehicle-mounted multi-language speech recognition method and system, electronic equipment and a storage medium, and relates to the field of cockpit speech control, and the method comprises the steps: collecting multi-language audio data of different acoustic environments in a cockpit; preprocessing the multi-voice audio data, extracting voice features, and outputting feature vectors; acquiring a multi-language speech recognition engine, and training the recognition engine according to the feature vector; obtaining target audio data, performing recognition through the trained multi-language speech recognition engine, and generating a recognition result; obtaining a verification mechanism, verifying the identification result according to the verification mechanism, and generating a verification result; generating a correction result set according to the verification result; and correcting the multi-language speech recognition engine according to the correction result set, and reducing the noise interference and increasing the robustness of speech recognition through a mode of firstly performing speech enhancement and then performing feature extraction.

This invention belongs to the field of text printing technology and discloses a text printing method and system based on speech recognition. The method includes the following steps: constructing a text printing template database; constructing a mixed-language speech recognition model; acquiring speech audio data in real time and performing speech recognition; matching several corresponding text printing templates; selecting a text printing template; fusing the real-time speech text data with the selected text printing template; and printing the real-time text printing data. The system includes a database construction unit, a model construction unit, a storage unit, a speech audio acquisition unit, a speech recognition application unit, a template matching unit, a template selection unit, a data fusion unit, and a printer. This invention solves the problems of low intelligence, poor speech recognition effect, low recognition efficiency, and lack of organic integration in existing technologies.

The invention provides a speech translation method, a server, a storage medium and a program product. The method relates to the field of artificial intelligence. The method comprises the following steps: acquiring original voice of a to-be-translated original language and a to-be-translated target language; and searching translation knowledge matched with the original voice in a translation knowledge base, inputting the original voice and the translation knowledge matched with the original voice into a voice translation model, translating the original language voice in the translation knowledge appearing in the original voice into a corresponding target language text through the voice translation model, and generating a translation text of the target language. On the basis of a cross-modal translation knowledge base, external knowledge intervention is carried out on the speech translation model, translation errors of customized vocabularies are effectively avoided on the premise that parameters of the speech translation model do not need to be retrained, the translation accuracy of the speech translation model to customized vocabularies such as proper nouns, terminologies and new words is improved, and the translation efficiency is improved. Therefore, the accuracy of the speech translation result is improved.

The invention discloses a Chinese language pronunciation intonation evaluation method, and belongs to the technical field of speech processing. The method comprises the following steps: firstly, dividing a Chinese language speech signal by adopting an autocorrelation method to obtain a plurality of sound state sets; eMD decomposition is carried out on short-time frames in the sound state set, and four characteristics of low-frequency gravity center, low-frequency energy, high-frequency gravity center and high-frequency energy are extracted; for a single sound state set, obtaining a sound state instability coefficient and constructing a sequence based on the four EMD features of each short-time frame, and meanwhile, respectively averaging each EMD feature; obtaining a change rate according to the EMD feature mean value difference of the adjacent sound state sets, constructing four change rate sequences, and subtracting each change rate sequence from the corresponding change rate storage sequence to obtain a change rate difference sequence; and finally, processing the four change rate difference sequences by using a pronunciation intonation evaluation neural network, and obtaining a pronunciation intonation score by combining the pronunciation intonation evaluation neural network with the sound state instability coefficient sequence. The precision of Chinese language pronunciation intonation evaluation is improved.

The invention provides a language information dynamic detection method based on a deep neural network, and relates to the technical field of voice processing, and the method comprises the steps: obtaining a to-be-detected mixed language voice stream, and extracting original acoustic features; generating an attention vector of the current time step based on an attention mechanism, and dynamically selecting a voice feature frame sequence window of the current time step in combination with unidirectional time selection and limitation of a specific time span; multiplying the dynamic window signal by the original acoustic feature signal to generate a local acoustic feature of the current time step detection language information; normalizing the local acoustic features of any length into fixed dimension features, inputting the fixed dimension features into a deep neural network classifier, and outputting a language probability value corresponding to the voice features of the current time step; based on the dynamic window information and the language probability value, the starting and ending time of each voice segment in the mixed voice stream and the corresponding language label are output, the time point of language switching is determined, and the method is particularly suitable for language switching recognition in the bilingual mixed voice stream.

The invention provides a voice conversion method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the extraction of content features and fundamental frequency features of a source audio, and carrying out the extraction of speaker features of a target audio; inputting the content features, the fundamental frequency features and the speaker features into a voice conversion model for joint modeling processing, linear mapping processing and waveform reconstruction processing to generate a voice waveform; performing one-dimensional deep separation convolution processing and multi-receptive-field fusion processing on the voice waveform based on a vocoder in the voice conversion model to generate converted voice audio; wherein the converted voice audio shows that the speaking style of the source audio is replaced by the speaker style of the target audio. The voice conversion model is used for voice conversion, so that the timbre consistency and semantic retention capability of cross-language voice conversion are improved, and the naturalness and definition of reconstructed voice are improved.

This disclosure relates to a speech synthesis method, apparatus, medium, and electronic device. The method includes: acquiring a phoneme sequence of a text to be synthesized and target prosodic information corresponding to the phoneme sequence, wherein the target prosodic information is prosodic information under a first language to which the text to be synthesized belongs; and synthesizing target speech based on the phoneme sequence, the target prosodic information, a speech vector of a first speaker under the first language, and a timbre vector of a second speaker, wherein the target speech represents the speech of the second speaker speaking the text to be synthesized under the first language. The speech synthesis method of this disclosure can effectively solve the problem of cross-language speech synthesis, and the synthesized speech has high pronunciation accuracy and natural prosody.