Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

97 results about "Phonetic representation" patented technology

Phonetic representation, or more commonly phonetic transcription is the representation of speech sounds using symbols in phonetic alphabet such as IPA, X-SAMPA, Kirshenbaum for linguistic studies and for learning the pronunciation of languages. Among these systems, the International Phonetic Alphabet has been the most widely used so far, whose symbols are printed in most dictionaries and books on linguistics.

Low-Latency Conversational Large Language Models

A method includes receiving a transcription of an utterance, processing, using a first model, the transcription to generate a first text segment that represents an initial portion of a response to the utterance, processing, using a TTS system, the first text segment to generate a first synthesized speech representation, and providing, for audible output, the first synthesized speech representation. The method also includes providing, to a second model different from the first model, the transcription and the first text segment, the second model comprising an LLM configured to process the transcription and the first text segment to generate a second text segment that represents a remaining portion of the response to the utterance. The method further includes obtaining a second synthesized speech representation generated from the second text segment, and providing, for audible output by the user device, the second synthesized speech representation.
Owner:GOOGLE LLC

Feature attention and bilinear gating fused speech emotion recognition method and device

The invention discloses a feature attention and bilinear gating fused speech emotion recognition method and device, and the method comprises the following steps: 1, collecting an audio file, obtaining corresponding label information, generating audio waveform and time frequency representation data through preprocessing, and constructing an audio waveform mask and a time frequency mask to mark an effective information region; 2, constructing a dual-path feature extraction module which comprises a time-frequency feature extraction module and a pre-training acoustic feature coding module; wherein the time-frequency feature extraction module models emotion correlation through local convolution and a multi-dimensional attention mechanism, and performs global time sequence modeling based on a bidirectional gated loop network; the pre-training acoustic feature coding module extracts high-level speech representation with high expression ability for emotion distinguishing by using a pre-training model; and step 3, constructing a feature fusion module and an emotion classification module, and combining with a dual-path feature extraction module to form a speech emotion recognition model.
Owner:SICHUAN UNIV

Audio and video depth forgery detection method based on quality perception and multi-scale alignment

The invention discloses an audio and video depth forgery detection method based on quality perception and multi-scale alignment, and the method comprises the following steps: coding a synchronous audio and video sequence, and obtaining a frame-level visual feature, a facial action unit and a phoneme-level voice representation; a visual quality evaluation module is introduced to generate a spatial reliability mask, and quality weighting is carried out on the visual features; designing a global-local multi-scale cross-modal alignment mechanism, performing bidirectional cross-attention modeling on voice and face dynamic synchronization globally, and performing physiological coupling alignment on phonemes and face action units locally; and an uncertainty perception reasoning and calibration scheme is provided, adaptive temperature scaling is carried out according to quality and consistency, and uncertainty calibration is carried out by self-supervision loss. According to the method, the problems of insufficient robustness and excessive self-confidence misjudgment of an existing method in a low-quality video and high-synchronization counterfeit scene are solved, and the cross-dataset generalization capability and the actual deployment reliability are remarkably improved.
Owner:NANJING UNIV OF SCI & TECH

Law enforcement recorder audio semantic retrieval method and system based on retrieval enhancement

The invention relates to the technical field of speech recognition and natural language processing, and particularly discloses a law enforcement recorder audio semantic retrieval method and system based on retrieval enhancement, and the system comprises a data acquisition module which is used for acquiring audio data and text query; the voice adapter module is used for projecting the audio data to a text embedding space to obtain voice representation; the cross-modal retriever is used for carrying out cross-modal retrieval on the voice representation and the text query to obtain voice Tokens; the voice language model is used for obtaining character hypotheses; the query generation module is used for extracting query fragments possibly containing entity names; the entity retrieval module is used for retrieving an entity database according to the query fragment to obtain a related entity name; the context construction module is used for constructing context information; and the large language model is used for obtaining a semantic retrieval result. According to the method, the identification and retrieval accuracy of the entity name and key information in the law enforcement recorder audio is improved.
Owner:先进计算与关键软件(信创)海河实验室 +1

Unsupervised data selection via discrete speech representations for automatic speech recognition

A method (500) includes obtaining an untagged training data corpus (358) including a plurality of spoken utterances (360), each corresponding spoken utterance of the plurality of spoken utterances including audio data (362) characterizing the corresponding spoken utterance. The method also includes receiving a target domain (324). The method further includes using a contrast data selection model (310) to select a subset of utterances corresponding to the target domain from the untagged corpus of training data. The method includes training an automatic speech recognition (ASR) model on a subset of utterances (200).
Owner:GOOGLE LLC

System and Method for Secure Processing of Speech Signals using Pseudo-speech Representations

A method, computer program product, and computing system for processing a speech signal. A sensitive portion of the speech signal is identified. A pseudo-speech representation of the sensitive portion is generated using a voice converter system. Speech processing is performed on the speech signal and the pseudo-speech representation of the sensitive portion using a speech processing system.
Owner:MICROSOFT TECHNOLOGY LICENSING LLC

Scaling Multilingual Speech Synthesis with Zero Supervision of Found Data

A method includes receiving training data that includes a plurality of sets of training utterances each associated with a respective language. Each training utterance includes a corresponding reference speech representation paired with a corresponding input text sequence. For each training utterance, the method includes generating a corresponding encoded textual representation for the corresponding input text sequence, generating a corresponding speech encoding for the corresponding reference speech representation, generating a shared encoder output, and determining a text-to-speech (TTS) loss based on the corresponding encoded textual representation, the corresponding speech encoding, and the shared encoder output. The method also includes training a TTS model based on the TTS losses determined for the training utterances in each set of the training utterances to teach the TTS model to learn how to synthesize speech in each of the respective languages.
Owner:GOOGLE LLC

Joint unsupervised and supervised training for multilingual ASR

ActiveUS12249317B2Pattern recognitionMedicine
A method includes receiving audio features and generating a latent speech representation based on the audio features. The method also includes generating a target quantized vector token and a target token index for a corresponding latent speech representation. The method also includes generating a contrastive context vector for a corresponding unmasked or masked latent speech representation and deriving a contrastive self-supervised loss based on the corresponding contrastive context vector and the corresponding target quantized vector token. The method also include generating a high-level context vector based on the contrastive context vector and, for each high-level context vector, learning to predict the target token index at the corresponding time step using a cross-entropy loss based on the target token index. The method also includes predicting speech recognition hypotheses for the utterance and training a multilingual automatic speech recognition (ASR) model using an unsupervised loss and a supervised loss.
Owner:GOOGLE LLC

Large model real-time voice interaction method and device based on autoregression voice synthesis

The invention provides a large-model real-time voice interaction method and device based on autoregressive voice synthesis, and the method comprises the steps: obtaining a voice instruction marked with a target text response and a target voice response, enabling a voice encoder to code the voice instruction into voice representation, and enabling a voice adapter to carry out the dimension reduction and feature conversion of the original voice representation; the large language model generates a hidden state according to the converted voice representation and samples the hidden state to obtain a text sequence; and processing the text sequence by adopting a text-voice language model based on an autoregression Transform structure, generating a voice marking sequence in a streaming manner, and converting the voice marking sequence into a voice signal through a vocoder. According to the method provided by the invention, the naturalness and fluency of speech synthesis are greatly improved while high real-time performance is ensured. The optimized voice decoding architecture effectively reduces the voice generation delay and improves the response speed of the voice interaction system.
Owner:INST OF COMPUTING TECH CHINESE ACAD OF SCI

Realtime AI sign language recognition with avatar

Disclosed herein are method and system aspects for translating between a sign language and a target language and presenting such translations. For example, a method receives input language data and translates the input language data into sign language grammar. The method retrieves phonetic representations that correspond to the sign language grammar from a sign language database and generates coordinates from the phonetic representations using a generative network. The phonetic representations are digital representations of individual signs created through manual input of body configuration information corresponding to the individual signs. Further, the method renders an avatar that moves between the coordinates. In another example, a bidirectional communication system allows for realtime communication between a signing entity and a non-signing entity.
Owner:SIGN-SPEAK INC

Children voice expression error recognition and correction method based on comparative learning

PendingCN121011207ASpeech analysisSpeech developmentFalse recognition
The invention discloses a children voice expression error recognition and correction method based on comparative learning, and the method comprises the steps: carrying out the preprocessing of an inputted children voice signal, obtaining a logarithmic Mel spectrum feature sequence, and converting the logarithmic Mel spectrum feature sequence into voice semantic coding features through an improved Transform encoder; on the basis of an age-adaptive positive and negative sample selection mechanism, voice features are optimized by using a comparative learning method, and an enhanced voice representation vector is obtained; constructing a multi-modal fusion network, combining an enhanced voice vector and BERT language model features, realizing adaptive fusion through multi-head cross attention and a gating mechanism, adopting a bidirectional LSTM to design an error positioning module to recognize the position and the type of a pronunciation error, and using a joint loss function to execute end-to-end training; standard correction audio is generated according to the error type, and correction guidance is provided for children. According to the invention, high-precision recognition, positioning and correction of children's speech expression errors are realized, and technical support is provided for children's language development.
Owner:HECHEN ZONGHENG INFORMATION TECH CO LTD

Voice cloning model generation method and device, storage medium and computer equipment

The invention discloses a voice clone model generation method and device, a storage medium and computer equipment, relates to the technical field of voice synthesis and medical treatment, and mainly aims at solving the problems that a voice clone model trained by an existing method is relatively weak in noisy voice recognition capability and insufficient in robustness. The method mainly comprises the steps of obtaining a voice training sample set; a training model of the voice cloning model is constructed, and the training model of the voice cloning model comprises an initial speaker encoder, a self-supervised voice representation learning model and an initial voice synthesizer; training an initial speaker encoder and an initial voice synthesizer by using a natural voice training sample and a recognition result of a self-supervised voice representation learning model on the natural voice training sample; and extracting a speaker encoder and a voice synthesizer in the training model of the voice clone model, and generating the voice clone model according to the speaker encoder, the voice synthesizer and the text encoder. The method is mainly used for training a speech clone model.
Owner:PING AN TECH (SHENZHEN) CO LTD

Self-supervised speech representations for fake audio detection

A method for determining synthetic speech includes receiving audio data characterizing speech in audio data obtained by a user device. The method also includes generating, using a trained self-supervised model, a plurality of audio features vectors each representative of audio features of a portion of the audio data. The method also includes generating, using a shallow discriminator model, a score indicating a presence of synthetic speech in the audio data based on the corresponding audio features of each audio feature vector of the plurality of audio feature vectors. The method also includes determining whether the score satisfies a synthetic speech detection threshold. When the score satisfies the synthetic speech detection threshold, the method includes determining that the speech in the audio data obtained by the user device comprises synthetic speech.
Owner:GOOGLE LLC

System and method for automated processing of speech using machine learning

Automated systems and methods are provided for processing speech, comprising obtaining a digitally-encoded speech representation corresponding to a telecommunication interaction, wherein the digitally-encoded speech representation includes at least one of a voice recording or a transcript derived from audio of the telecommunication interaction; obtaining a digitally-encoded data set corresponding to at least one structured feature of the telecommunication interaction; obtaining a reference set, wherein the reference set includes a set of binary-classified existing satisfaction classifications; obtaining a trained machine learning algorithm, wherein the machine learning algorithm has been trained using a first plurality of reference telecommunication interactions which include user-provided satisfaction scores; extracting a feature set from the digitally-encoded speech representation; and by the machine learning algorithm and based on at least one structured feature and the feature set, generating a predicted satisfaction classification for the telecommunication interaction.
Owner:CONDUENT BUSINESS SERVICES LLC

Speech synthesis with foreign segments

A method for synthesizing speech based on a textual input includes receiving a textual input including native words of a native language and foreign words of a foreign language and processing the textual input to determine a phonetic representation of the textual input. The processing includes determining a native phonetic symbol representation of the native word and determining a localized phonetic symbol representation of the foreign word. Determining the localized phonetic representation includes forming a foreign phonetic representation of the foreign word using the set of foreign phonemes, and mapping the foreign phonetic representation to the localized phonetic representation according to a model of pronunciation of the foreign word by the local speaker.
Owner:CERENCE OPERATING CO

Low-latency conversational large language models

A method includes receiving a transcription of an utterance, processing, using a first model, the transcription to generate a first text segment that represents an initial portion of a response to the utterance, processing, using a TTS system, the first text segment to generate a first synthesized speech representation, and providing, for audible output, the first synthesized speech representation. The method also includes providing, to a second model different from the first model, the transcription and the first text segment, the second model comprising an LLM configured to process the transcription and the first text segment to generate a second text segment that represents a remaining portion of the response to the utterance. The method further includes obtaining a second synthesized speech representation generated from the second text segment, and providing, for audible output by the user device, the second synthesized speech representation.
Owner:GOOGLE LLC

Control device, handling system, determining device, control method and computer program

UndeterminedDE102026106796A1Object basedEngineering
According to one embodiment, a control device (3; 3-2) recognizes an object area from image information that corresponds to an object as a target in automated handling operations. The control device generates a handling strategy for the object and outputs a speech representation of the handling strategy. The control device constructs the object area based on a speech representation of an object shape encompassed in the handling strategy. The control device evaluates the handleability of the object based on geometric information about the object area. The control device evaluates the adaptability of a handling tool (14) to the object based on the speech representation of the object shape.Based on the manageability and adaptability of the handling tool (14), the control device determines whether handling of the object is possible and / or generates a handling position and a posture of the handling tool (14).
Owner:TOSHIBA KANAGAWA

Robust direct speech-to-speech translation

A direct speech-to-speech translation (S2ST) model (200 includes an encoder (210) configured to receive an input speech representation (102) that to an utterance (108) spoken by a source speaker (104) in a first language and encode the input speech representation into a hidden feature representation (215). The S2ST model also includes an attention module (220) configured to generate a context vector (225) that attends to the hidden representation encoded. The S2ST model also includes a decoder (230) configured to receive the context vector generated by the attention module and predict a phoneme representation (235) that corresponds to a translation of the utterance in a second different language. The S2ST model also includes a synthesizer (300) configured to receive the context vector and the phoneme representation and generate a translated synthesized speech representation (355) that corresponds to a translation of the utterance spoken in the different second language.
Owner:GOOGLE LLC

Techniques for disentangled variational speech representation learning for zero-shot voice conversion

A method for disentangled variational speech representation learning for voice conversion, performed by at least one processor, is provided. The method includes receiving input speech segments, encoding the input speech segments via a shared encoder to generate a speaker embedding and a content embedding, encoding the posterior distributions of the speaker embedding via a speaker encoder and encoding the posterior distributions of the content embedding via a content encoder to obtain encoded results, and decoding the encoded results by concatenating the encoded results to obtain a reconstructed speech output.
Owner:TENCENT AMERICA LLC

Controllable zero sample voice conversion method, device, equipment and medium

The invention relates to the technical field of voice semantics, can be applied to business system platforms of financial science and technology, medical treatment and health and the like, and discloses a controllable zero sample voice conversion method, device, equipment and medium, the method comprises the following steps: carrying out self-supervised voice learning on unlabeled voice data to obtain self-supervised voice representation; the method comprises the following steps of: extracting a content feature vector and a rhythm style vector represented by self-supervised speech, converting the content feature vector and the rhythm style vector into a discrete content token and a discrete rhythm token, performing mask generation on the discrete rhythm token to obtain a target rhythm token, obtaining reference speech of a target user, extracting user style embedding in the reference speech, and obtaining a target user. And performing stream matching on the discrete content token, the target rhythm token and the user style embedding to generate a target Mel spectrogram, and performing voice waveform reconstruction and optimization on the target Mel spectrogram to obtain a zero sample voice conversion result. According to the invention, under the condition of no annotated voice data, personalized, high-fidelity and style-consistent zero-sample voice conversion is realized.
Owner:PING AN TECH (SHENZHEN) CO LTD

Training method and system for self-supervised training predictor for speech separation

ActiveCN115762557Bimprove performanceSelf-supervised training feature improvementSpeech recognitionSound sourcesVoice source
The embodiment of the present application provides a training method and system of a self-supervised training predictor for speech separation. The method comprises: extracting self-supervised training features of each single person voice source speech by using a pre-training model; extracting shallow features for speech representation and deep features for context information in the self-supervised training features, and determining the shallow features and the deep features of each single person voice source speech as training labels of the self-supervised training predictor; inputting training mixed speech generated by each single person voice source speech into the self-supervised training predictor to obtain estimated features of each single person voice source speech; and training the self-supervised training predictor based on a loss function determined based on the estimated features and the training labels corresponding to each single person voice source speech. The self-supervised training predictor is trained and applied in a speech separation model in the embodiment of the present application, so that the accuracy of the self-supervised training features is improved, the performance of the speech separation system is improved, and the model parameters and the calculation complexity are reduced.
Owner:AISPEECH CO LTD

Emotive text-to-speech with auto detection of emotions

A method of providing emotive text-to-speech includes obtaining input text characterizing a natural language response generated by an assistant LLM to a query input by a user during a conversation between the user and the assistant LLM, and processing, using the assistant LLM, the input text conditioned on an emotion detection task prompt to predict, as output from the assistant LLM, an emotional state of the natural language response. The method also includes determining, based on the emotional state of the natural language response predicted as output from the assistant LLM, an emotional embedding for the input text and instructing a TTS model to process the input text and the emotional embedding to generate a synthesized speech representation of the natural language response conveying the emotional state of the natural language response as specified by the emotional embedding.
Owner:GOOGLE LLC

Secure processing of speech signals using pseudo-speech representations

A method, computer program product, and computing system for processing a speech signal. A sensitive portion of the speech signal is identified. A pseudo-speech representation of the sensitive portion is generated using a voice converter system. Speech processing is performed on the speech signal and the pseudo-speech representation of the sensitive portion using a speech processing system.
Owner:MICROSOFT TECHNOLOGY LICENSING LLC

Audio Optimization Method and Related Devices, Electronic Devices, Storage Media

The present application discloses an audio optimization method and related devices, electronic devices, and storage media. Among them, the audio optimization method includes: extracting a first audio representation of the collected audio and extracting a second audio representation of the reference audio; respectively extracting a first echo representation, a first speech representation, and a first noise representation based on the first audio representation and the second audio representation; performing interactive processing on the first speech representation with the first echo representation and the first noise representation respectively to obtain a second speech representation, a second echo representation, and a second noise representation; where the interactive processing includes: echo suppression, noise suppression, and speech enhancement; obtaining an optimized target audio based on at least one of the second speech representation, the second echo representation, and the second noise representation. The above solution can improve the audio optimization effect.
Owner:IFLYTEK CO LTD

Method, apparatus, device and storage medium for text-to-speech conversion

According to embodiments of this disclosure, a method, apparatus, device, and storage medium for text-to-speech conversion are provided. The method includes generating a predicted speech representation of the target text read by a first speaker, based on a target text to be converted and a first timbre of a first speaker. The predicted speech representation indicates speech features that vary over time. The method further includes generating a predicted time-frequency representation of the target text read by a second speaker, based on the predicted speech representation and a second timbre of a second speaker. The predicted time-frequency representation indicates the speech signal strength that varies over time at different frequencies. The method further includes converting the predicted time-frequency representation into audio of the target text read by the second speaker. This reduces the difficulty of prediction and improves the sound quality of the generated audio.
Owner:BEIJING YOUZHUJU NETWORK TECH CO LTD

Hybrid language models for conversational AI systems and applications

In various examples, first textual data may be applied to a first MLM to generate an intermediate speech representation (e.g., a frequency-domain representation), the intermediate audio representation and a second MLM may be used to generate output data indicating second textual data, and parameters of the second MLM may be updated using the output data and ground truth data associated with the first textual data. The first MLM may include a trained Text-To-Speech (TTS) model and the second MLM may include an Automatic Speech Recognition (ASR) model. A generator from a generative adversarial networks may be used to enhance an initial intermediate audio representation generated using the first MLM and the enhanced intermediate audio representation may be provided to the second MLM. The generator may include generator blocks that receive the initial intermediate audio representation to sequentially generate the enhanced intermediate audio representation.
Owner:NVIDIA CORP

System and method for automated processing of digitized speech using machine learning

Automated systems and methods are provided for processing speech, comprising obtaining a trained machine learning model that has been trained using a cumulative historical data structure corresponding to at least one digitally-encoded speech representation for a plurality of telecommunications interactions conducted by a plurality of agent-side participants, which includes a first data corresponding to a score variable and a second data corresponding to a plurality of driver variables; applying the trained machine learning model: to a subset of data in the cumulative historical data structure that corresponds to a first agent-side participant of the plurality of agent-side participants, to generate a performance classification score and / or a performance direction classification score, to identify an intervention-target agent-side participant from among the plurality of agent-side participants, and to the cumulative historical data structure to identify an intervention training plan; and conducting at least one training session according to the intervention training plan.
Owner:CONDUENT BUSINESS SERVICES LLC

Voice authentic identification method and device, computer equipment and storage medium

The invention relates to the technical field of artificial intelligence, finance and medical health, and discloses a voice authentication method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring to-be-identified voice information; inputting the voice information to be subjected to authentic identification into the self-supervised large model for feature extraction to obtain an extraction result; inputting the potential speech representation features into a classification model for deep coding to obtain a coding result, converting the coding result into low-dimensional representation, and calculating the probability distribution of speech authenticity to obtain a classification result; and outputting a classification result. By implementing the method provided by the invention, the capability of identifying the forged voice in high-imitation audio and noise environments can be remarkably improved, and reliable guarantee is provided for voice safety in key fields such as finance, medical health and the like.
Owner:PING AN TECH (SHENZHEN) CO LTD

A method and system for audio semantic retrieval of law enforcement recorders based on retrieval enhancement

The present invention relates to the technical fields of speech recognition and natural language processing, and specifically discloses a method and system for semantic retrieval of law enforcement recorder audio based on retrieval enhancement. The system comprises: a data acquisition module for acquiring audio data and text queries; a speech adapter module for projecting the audio data into a text embedding space to obtain a speech representation; a cross-modal retriever for performing cross-modal retrieval of the speech representation and text query to obtain speech tokens; a speech language model for obtaining text hypotheses; a query generation module for extracting query fragments that may contain entity names; an entity retrieval module for searching an entity database based on the query fragments to obtain relevant entity names; a context construction module for constructing contextual information; and a large language model for obtaining semantic retrieval results. The present invention improves the recognition and retrieval accuracy of entity names and key information in law enforcement recorder audio.
Owner:先进计算与关键软件(信创)海河实验室 +1

Realtime ai sign language translation with avatar

Disclosed herein are method and system aspects for translating between a sign language and a target language and presenting such translations. For example, a method receives input language data and translates the input language data into sign language grammar. The method retrieves phonetic representations that correspond to the sign language grammar from a sign language database and generates coordinates from the phonetic representations using a generative network. The phonetic representations are digital representations of individual signs created through manual input of body configuration information corresponding to the individual signs. Further, the method renders an avatar that moves between the coordinates. In another example, a bidirectional communication system allows for realtime communication between a signing entity and a non-signing entity.
Owner:SIGN-SPEAK INC