1387 results about "Speech processing" patented technology

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses an audio signal authenticity verification method, device, equipment and medium, and the method comprises the steps: constructing an original audio text data set, and generating an adversarial sample set, inputting the original audio text data set and the adversarial sample set into an audio detection model for joint training to obtain an audio detection model subjected to adversarial training; the method comprises the steps of obtaining a to-be-detected audio signal and extracting an acoustic feature of the to-be-detected audio signal, obtaining a non-acoustic feature associated with the to-be-detected audio signal, constructing a multi-dimensional feature vector according to the acoustic feature and the non-acoustic feature, inputting the multi-dimensional feature vector into an audio detection model to generate an abnormal index, and executing a hierarchical response operation based on the abnormal index. According to the method, the robustness of the model is enhanced by introducing adversarial sample training, and the multi-dimensional feature vector is constructed by fusing the multi-modal features, so that accurate recognition and hierarchical response to the voice cloning attack are realized.

The invention relates to a character-driven lip sound synchronous digital human generation method and device, equipment and a medium, and the method comprises the steps: obtaining an input text and a single object video, and carrying out the preprocessing of the input text, and obtaining a phoneme sequence and semantic features; the method comprises the following steps: preprocessing a single-object video to obtain a circularly played video paragraph, a mask face image and a reference face image; generating voice features based on the phoneme sequence and the semantic features; generating an audio sequence based on the voice features; acquiring face deformation parameters according to the mask face image, the reference face image and the voice features, and deforming the reference face image to generate a target face image; and embedding the target face image into a video frame of the video paragraph to generate a video sequence, and splicing the video sequence with the audio sequence to obtain a target digital human video. The digital human video is generated based on text driving, so that the voice and lip-shaped video matched with the text can be generated in time, the time delay for waiting for text to be converted into voice processing is eliminated, and the digital human video production efficiency is greatly improved.

Provided in the embodiments of the present application are a video generation method and interaction method based on a digital human, and a device, a storage medium and a program product. In the embodiments of the present application, text-to-speech processing is performed on the basis of voice features of a user and an emotion label, speech-to-expression processing is performed on the basis of a mapping relationship between the voice features of the user and expression coefficients, and a digital human model is rendered on the basis of speech signals and the expression coefficients, so as to obtain video data of the digital human model. Thus, voice features of a user are accurately simulated, so as to ensure that a speech output of a digital human sounds natural and is also highly personalized, thereby realizing personalized driving of the digital human, and improving the realism of the digital human in terms of voice and dynamic images. Thus, the user experience is improved, and the interactivity of the digital human and the authenticity and immersion are enhanced.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice feature processing method, device, equipment and medium. Performing time resolution analysis based on the fused Mel band energy to generate a multi-scale Mel spectrum amplitude value, and performing nonlinear transformation on the multi-scale Mel spectrum amplitude value according to the noise intensity parameter to generate a noise suppression Mel component; and generating a perception weighting coefficient according to an auditory perception model, and executing frequency domain energy adjustment on the noise suppression Mel component to generate Mel spectrum representation. On the basis of frequency resolution self-adaption, time resolution dynamic adjustment and auditory perception modeling, nonlinear transformation and perception weighting processing are applied to the multi-scale Mel spectrum amplitude value, the influence of noise interference on voice features can be effectively reduced, and the key information retention capacity of voice signals is enhanced.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses an audio coding and decoding method, device, equipment and medium, and the method comprises the steps: carrying out the sliding window segmentation processing of an input audio signal, and generating signal segments; processing the signal segments through an encoder containing a multi-layer self-attention mechanism to generate continuous potential representations; performing decomposition vector quantization processing on the continuous potential representation to generate a discrete code; the discrete codes are processed through a decoder comprising a multi-layer self-attention mechanism, and reconstructed signal segments are generated; and splicing the reconstructed signal segments to generate a complete audio signal. According to the method, a traditional convolution structure is replaced by a multi-layer self-attention mechanism, global time sequence dependence modeling is carried out on the audio signals after sliding window segmentation, effective compression of potential representation is realized in combination with decomposition vector quantization, and continuity and fidelity of audio reconstruction are improved on the premise that calculation complexity is not increased.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice enhancement method, device and equipment based on noise perception and a medium. Environment feature information is extracted and input into an audio enhancement model to generate an enhanced audio signal; obtaining a reference audio sample, extracting a personalized feature vector, and carrying out personalized processing on the enhanced audio signal; and collecting playing feedback data, determining a playing time domain adjustment parameter and a playing frequency domain adjustment parameter, adjusting the personalized enhanced audio signal, and generating an optimized audio signal. According to the method, dynamic adjustment is realized in combination with the feedback parameters in the playing process by fusing the environmental perception information and the personalized speaker characteristics, clear and natural optimized audio output with personalized styles can be generated in a complex environment, and the voice interaction quality and adaptability are improved.

An intelligent (self-learning) sensor-aware and / or context-aware subsystem comprising (i) a System-on-a-Chip (SoC), (ii) a radio transceiver, (iii) a microphone, (iv) a voice processing module, (v) a (bio-inspired) neuromorphic event camera or a hyperspectral camera, (vi) a first set of computer implemental instructions in artificial neural networks (ANN) (which may include a transformer model / diffusion model or Poisson flow generative model++ (PFGM++)), (vii) a second set of computer implementable instructions to analyze and interpret contextual data and (viii) an autonomous artificial intelligence (AI) agent is disclosed.

The invention relates to the technical field of voice processing, can be applied to service system platforms of medical health, financial science and technology, communication and the like, and discloses an audio compression and reconstruction method, device, equipment and medium. Residual vector quantization and inverse quantization are carried out to generate basic spectrum features; and compensating a high-frequency component by adopting a spectrum expansion network, predicting a time domain error in combination with a long-short-term memory network, generating a time domain residual compensation signal, and superposing the time domain residual compensation signal with the reconstructed time domain audio signal to obtain a target audio signal. Through residual vector quantization and neural network modeling, the audio reconstruction quality under low-bit-rate compression is improved, spectrum detail reservation and time domain error compensation are optimized, meanwhile, the calculation complexity is reduced, and the method is suitable for high-sampling-rate and resource-limited scenes.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health, voice navigation and the like, and discloses a voice processing method, device, equipment and medium based on voiceprint feature screening. The method comprises the following steps: screening voice of a near-field main speaker based on voiceprint similarity and signal intensity, executing duration filtering and confidence verification, dynamically constructing a voiceprint feature library, generating a voiceprint mask matrix based on the voiceprint feature library, performing frequency band suppression on a to-be-processed voice signal, and outputting a purified voice signal. According to the method, the voiceprint library is dynamically constructed, the voiceprint mask matrix is generated based on the voiceprint feature library, frequency band suppression is performed on the to-be-processed voice signal, non-target voiceprints are effectively shielded, the voice signal quality is improved, and thus target voice is accurately captured in a high-noise environment.

The invention relates to the technical field of voice processing, can be applied to business scenes in the fields of medical health, financial science and technology, barrier-free service and the like, and discloses a text-to-voice method based on knowledge distillation, which comprises the following steps: carrying out standardization processing on an input text to generate a standard text sequence; the lightweight text encoder encodes the standard text sequence to generate a text implicit vector; the non-autoregressive acoustic feature prediction module maps the text implicit vector into a student acoustic feature sequence, and calculates alignment loss through knowledge distillation; and performing structured pruning and parameter quantization based on alignment loss, generating an acoustic feature sequence by the optimized model, and converting the acoustic feature sequence into a voice waveform by a vocoder. According to the method, through knowledge distillation, pruning optimization and parameter quantification, the reasoning speed and the cross-equipment adaptability are improved while the model size and the calculation requirement are reduced, so that the TTS system can realize efficient, low-delay and low-power-consumption voice generation in a resource-constrained environment.

A method of presenting a signal to a speech processing engine is disclosed. According to an example of the method, an audio signal is received via a microphone. A portion of the audio signal is identified, and a probability is determined that the portion comprises speech directed by a user of the speech processing engine as input to the speech processing engine. In accordance with a determination that the probability exceeds a threshold, the portion of the audio signal is presented as input to the speech processing engine. In accordance with a determination that the probability does not exceed the threshold, the portion of the audio signal is not presented as input to the speech processing engine.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a context information-based voice emotion recognition method, device, equipment and medium, which comprises the following steps: receiving an original voice stream and generating an independent voice segment, recognizing a text and determining a speaker role type, and extracting an acoustic feature index; and generating a preliminary emotion label, generating context information in combination with the historical dialogue text, and inputting the context information, the preliminary emotion label, the speaker role type and the acoustic feature index into a multi-modal fusion module to generate an emotion judgment result. According to the method, multi-modal fusion is realized on the basis of context information by combining voice, text and role information, so that the emotion change of each role can be accurately recognized and understood in a complex dialogue scene, the problems of large single sentence emotion judgment error and neglect of the context information in a traditional method are avoided, and the accuracy and stability of emotion recognition are effectively improved.

A device capable of using image data for purposes of determining a location of a user and audio beam selection to isolate audio in the direction of the user. The beamforming / beam-steering may occur after determining the user's location in order to conserve computing resources that would otherwise have been spent determining beams for non-desired directions. The beamformed audio may be used for speech processing, a communication session involving the device, or other purposes.

The invention relates to the technical field of speech processing, can be applied to business scenes of medical health, financial science and technology and the like, and discloses a speech enhancement method based on multi-scale feature learning, which comprises the following steps: framing an input audio signal, extracting a Mel-frequency spectrum feature, extracting a frequency domain feature by using a multi-scale convolutional neural network, and carrying out multi-scale feature learning on the frequency domain feature; carrying out coding dimension reduction on the image; noise is suppressed through a deep residual network, enhanced audio features are generated, a non-autoregression generative model is adopted for feature conversion, and finally a generative adversarial network is used for reconstructing a target voice waveform. Voice frequency domain features are extracted through the multi-scale convolutional neural network, and the feature expression ability of different frequency bands is improved; noise suppression is carried out through a deep residual network, and the purity of the voice signals is enhanced; feature conversion is optimized through a non-autoregression generation model, and the modeling efficiency of speech enhancement is improved; the target voice waveform is reconstructed through the generative adversarial network, and the naturalness and definition of the generated voice are improved.

The invention relates to the technical field of voice processing, can be applied to business scenes such as financial science and technology and medical health, and discloses an environmental acoustics simulation voice generation method, device and equipment and a medium, and the method comprises the steps: obtaining and separating mixed voice data, and generating original voice content and original environmental acoustics information; converting the original voice content into first text information; determining a target environment acoustic tag in combination with the original environment acoustic tag, the first text information and the target geographical location information; acquiring target environmental acoustic information from a preset sound data set based on the tag, and adjusting the amplitude characteristic of the target environmental acoustic information to match the original environmental acoustic information; and synthesizing the adjusted target environment acoustic information and the original voice content into simulated voice data. According to the method, the target geographic position information is introduced to participate in acoustic feature determination and amplitude adjustment, so that the generated simulated voice data is more consistent in geographic semantics and acoustic performance, and the authenticity and concealment of voice environment disguise are effectively improved.

Approaches are generally described for corrupted speech detection in voice-based computer interfaces. First input data including first audio data representing a user utterance may be received. First data representing the first audio data may be generated using a first encoder. First text data representing a transcription of the user utterance may be generated. Second data representing the first text data may be generated using a second encoder different from the first encoder. Third data may be generated by combining the first data and the second data. The third data may be sent to a classifier network trained to predict a relevant corruption state for speech processing inputs. The classifier network may determine that the first input data corresponds to a first corruption state.

An ear-worn device is provided that operates to isolate and individually treat the received speech of a target speaker or multiple target speakers from an audio input signal detected in a multi-speaker environment. The ear-worn device uses a machine learning model that receives a voice signature of each of one or more target speakers as input signals, to identify and isolate the component of the audio input signal attributable to the target speaker(s). Once isolated, the target speaker's speech may be enhanced, de-emphasized, or otherwise processed in a manner desired by the wearer of the ear-worn device. The wearer may use an external electronic device, e.g., a phone, to select one or more target speakers in a conversation and / or configure various settings associated with processing the speech on the ear-worn device.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice style migration method, device, equipment and medium, comprising the following steps: acquiring a source voice signal and a target style feature, performing feature extraction on the source voice signal to generate a source voice content feature and a potential style feature, encoding the target style feature to generate an encoded target style feature, performing style decoupling and migration processing on the source voice content feature, the potential style feature and the encoded target style feature by using a pre-trained multi-modal large model, and generating a migrated feature, and generating a target voice signal based on the migrated features. According to the method, the semantic and style information of the source voice is fused, the encoded target style features are combined to execute style migration, effective decoupling and adaptive fusion of contents and styles are realized by using a multi-modal large model, and the cross-speaker and cross-scene voice migration effect and practicability are improved.

The invention relates to the technical field of voice processing, in particular to an intelligent conference content real-time translation method based on voice recognition, which comprises the following steps of: acquiring an original voice signal, generating an enhanced voice stream by adopting a sudden change noise suppression algorithm, and synchronously extracting a voiceprint fingerprint spectrum; inputting the enhanced voice stream into a hierarchical separation network, performing multi-speaker voice decoupling based on a voiceprint fingerprint spectrum, outputting a voice segment with an identity tag, and triggering updating of an incremental term knowledge base; generating a cross-language semantic consistency vector, and constructing a dynamically updated context memory pool at the same time; and converting the semantic consistency vector into a target language stream. According to the method, the spatial accuracy and semantic independence of voice decoupling are improved, a structured input basis is provided for subsequent semantic modeling and translation, and the method is particularly suitable for conference scenes with cross speaking and frequent overlapping speech streams.

The invention relates to the technical field of voice processing, in particular to a mixed voice processing method, electronic equipment and a computer readable medium. The method comprises the following steps: collecting mixed voice and environment influence parameters; carrying out bionic frequency domain analysis on the mixed voice to obtain low-frequency attenuation compensation characteristic data; performing multipath effect propagation analysis on the low-frequency attenuation compensation characteristic data through the environmental influence parameters to generate channel distortion data; performing time domain-frequency domain joint deconvolution processing on the mixed voice by using the channel distortion data to generate a direct sound component and a reflected sound component; performing adversarial training based on the direct sound component and the reflected sound component to generate anti-multipath speech enhancement data; and constructing a dynamic frequency compensation filter based on preset environmental acoustic characteristics. Through the multi-stage signal processing, frequency compensation and real-time optimization technology, the output quality of the mixed voice is improved.

Devices and techniques are generally described for generating user satisfaction data in a natural language processing system. In various examples, first user input data may be received by a natural language processing system. Behavioral data related to the first user input data may be determined. Natural language processing error data related to the first user input data may be determined. First response data corresponding to the first user input data may be determined. First response characteristic data related to the first response data may be determined. First user satisfaction data may be determined based at least in part on the first response characteristic data, the behavioral data, and the natural language processing error data.

A speech-processing system may determine potential endpoints in a user's speech. Such endpoint prediction may include determining a potential endpoint in a stream of audio data, and may additionally including determining an endpoint score representing a likelihood that the potential endpoint represents an end of speech representing a complete user input. When the potential endpoint has been determined, the system may publish a transcript of speech that preceded the potential endpoint, and send it to downstream components. The system may continue to transcribe audio data and determine additional potential endpoints while the downstream components process the transcript. The downstream components may determine whether the transcript is complete; e.g., represents the entirety of the user input. Final endpoint determinations may be made based on the results of the downstream processing including automatic speech recognition, natural language understanding, etc.

The invention discloses a speech semantic analysis method and device, and a storage medium. The method comprises the steps of determining a real-time speech recognition result corresponding to frame-by-frame input speech; inputting the real-time voice recognition result and the context information into a semantic integrity judgment model to judge whether the real-time voice recognition result forms a complete semantic unit or not; and determining a real-time semantic analysis result corresponding to the real-time voice recognition result based on the streaming semantic analysis engine and the context information under the condition that the real-time voice recognition result is detected to form a complete semantic unit. Therefore, by introducing a frame-by-frame real-time speech recognition mechanism and a semantic integrity discrimination model, a processing link of staged dependence and waiting in a traditional speech processing system is broken, and closer dynamic cooperation between speech recognition and semantic analysis is realized.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice text bidirectional conversion method, device, equipment and medium, and the method comprises the steps: respectively executing voice recognition or voice synthesis operation according to the type of input information; for the voice information, noise suppression parameters are generated in combination with the lip movement video data, noise reduction processing is executed, and the recognition accuracy is improved; for text information, a pre-generated speaker style vector is obtained, the vector is cited in the speech synthesis process to generate natural personalized speech, and lip movement information and tactile feedback which are synchronous with speech output are generated. According to the method, complex noise is suppressed by fusing lip movement data, personalized voice is generated by using the style vector, and lip movement and touch information is output, so that bidirectional real-time conversion of voice and text in a complex environment is realized, and recognition accuracy, voice naturalness and interaction synchronism are effectively improved.

The invention relates to the technical field of voice processing, can be applied to business scenes such as financial science and technology and medical health, and discloses a multi-speaker dialogue voice analysis method, device and equipment and a medium, and the method comprises the steps: obtaining a to-be-analyzed multi-speaker dialogue voice, determining a naturalness score based on an acoustic feature, and obtaining a multi-speaker dialogue voice analysis result; determining a semantic consistency score based on voice embedding and semantic embedding corresponding to a preset text, determining a speaker consistency score based on embedding of a plurality of speakers of the same speaker, determining an interaction rationality score based on voice alternate overlapping duration, determining a diversity score based on a variance of voice features, and fusing the scores, the comprehensive mass fraction is obtained. According to the invention, through quantitative evaluation of five dimensions of naturalness, semantic consistency, speaker consistency, interaction rationality and diversity, a comprehensive quality scoring system is established, so that the evaluation result simultaneously reflects voice fluency, content matching degree, identity stability, interaction rhythm rationality and feature richness.

The invention discloses an audio data compression method and device, electronic equipment and a storage medium, relates to the technical field of voice processing, can be applied to financial science and technology and medical health business scenarios, and comprises the following steps: obtaining target audio data to be compressed; and inputting the target audio data into an audio coding and decoding quantization compression model to obtain a reconstructed audio signal corresponding to the target audio data, the audio coding and decoding quantization compression model being obtained through audio adversarial training. In the model, feature extraction and compression can be performed on target audio data by using a hierarchical neural network of an encoder to obtain a low-dimensional potential feature vector; a residual vector quantization module performs discrete quantization processing on the low-dimensional potential feature vector through a multi-layer cascaded codebook to obtain a discrete quantization code; and performing audio waveform reduction processing on the discrete quantization code by using a decoder to obtain a reconstructed audio signal corresponding to the target audio data. According to the invention, audio high-fidelity compression can be realized, and audio reconstruction tone quality is improved.

The present application provides techniques for implementing a skill component, configured to perform an assessment of a user, as part of a speech processing system. The system may receive a natural language user input requesting assistance. The skill component may, using one or more machine learning models, determine at least one characteristic of the natural language input (e.g., lexical embedding, acoustic embedding, topic, tone, etc.). The skill component may determine state data for a present session, where the state data indicates a topic of the natural language user input and / or a user state associated with the natural language user input. The skill component may determine past state data of one or more past sessions, and generate a question to the user based on the state data for the natural language user input and the past state data.

Systems and methods for embedding-based LLM tuning include generating a first embedding of received user input data and utilizing a translation model trained to associate user input with device names to generate a second embedding that differs at least in part from the first embedding. Reference embeddings corresponding to devices associated with user account data may be generated and a subset of the reference embeddings that satisfy a threshold similarity to the second embedding may be determined. A large language model (LLM) configured to determine a response to the user input data may utilize data for a subset of devices that correspond to the subset of the reference embeddings for performing speech processing.

According to the embodiment of the invention, a voice processing method and device, equipment and a medium are provided. The method comprises the steps that wake-up word voiceprint features of wake-up word voice segments are extracted, the wake-up word voice segments are matched with wake-up words, and the wake-up word voiceprint features indicate a target speaker; dividing the target voice into at least one voice segment, wherein the target voice is collected after the wake-up word voice segment; extracting at least one voiceprint feature of the at least one voice segment; based on the similarity between the at least one voiceprint feature and the wake-up word voiceprint feature, determining whether the at least one voice segment is from a target speaker; and determining instruction word recognition of the at least one voice segment based on a determination result of whether the at least one voice segment is from the target speaker. Therefore, noise interference or voice interference of others can be reduced in voice wake-up and voice control applications.