407 results about "Speech quality" patented technology

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of medical health, financial science and technology, cultural transmission and the like, and discloses a voice generation method based on multi-modal fusion, which comprises the following steps: collecting audio data to extract timbre features, and training a field feature timbre generation model; analyzing text semantic recognition emotion information, adjusting speech synthesis parameters, combining personalized information to construct a parameter mapping table, fusing to generate a synthesis control parameter sequence, aligning the synthesis control parameter sequence with character labels, visual elements and background music data, driving a domain feature timbre generation model, and generating synthesis data of synchronous speech, text, vision and music. Domain timbres are generated through timbre feature training, speech expression is optimized by combining semantic analysis and emotion recognition, user requirements are matched based on personalized information, and time alignment is performed by fusing text, vision and music data, so that the synthesized speech has domain features, emotion adaptability and personalization, and the speech quality is improved. And the voice immersion and the information transmission capability are improved.

The invention discloses a video translation method and system based on artificial intelligence. The method relates to the technical field of video translation and comprises the following steps of original sound track extraction, target AI speaker adaptation, AI dubbing generation and mouth shape synchronization and video synthesis. According to the method, independent audio and video streams are obtained by adopting an audio and video separation technology, and multiple original sound tracks are extracted through a voice separation model; matching or generating an adaptive target AI speaker module in a preset tone library; converting the original language voice into a text, translating the text into a target language text, and synthesizing an AI dubbing audio track in combination with a target AI speaker module; and finally, the independent video stream and the multi-AI dubbing audio track are input into the mouth shape synchronization model to output a translated video, so that the timbre fitting degree, the voice quality and the voice consistency of the same speaker of AI dubbing are improved, and meanwhile, the resource utilization rate of video translation and the processing efficiency under batch tasks are improved. The problem that in the prior art, video translation is low in quality and efficiency is solved.

The invention discloses a voice noise reduction system in a multi-noise environment, and particularly relates to the technical field of voice noise reduction, which comprises a basic division module for dividing voice data into a plurality of basic voice segments with the same duration; the rationality analysis module judges whether segmentation points of adjacent fragments are reasonable or not, and if the segmentation points are not reasonable, the optimization marking module searches for new segmentation points for optimization; the loop module repeatedly verifies the optimization result until all the voice segments are reasonably segmented; the noise reduction module extracts voice and noise features, and performs noise reduction processing by using a noise reduction strategy selection model; the recombination module recombines the voice segments after noise reduction into complete voice output; according to the invention, the voice signal is divided into a plurality of segments, the most suitable noise reduction algorithm can be matched for each segment according to the noise feature of each segment, and the method can cope with a more complex and unstable noise environment through a dynamically adaptive noise reduction strategy, and guarantees that the voice quality is better improved in various scenes.

The invention discloses a remote speech enhancement transmission method and system based on semantic communication in the technical field of speech signal transmission processing, and the system comprises a local transmitting end which is used for carrying out the short-time Fourier transform of a to-be-enhanced noisy speech signal, and then carrying out the semantic feature extraction of the frequency spectrum of the noisy speech signal through a semantic encoder; carrying out dimension adjustment on the voice semantic features by using a channel encoder; and finally, transmitting to a remote receiving end through a channel. The remote receiving end is used for receiving the voice semantic feature signal transmitted through the channel; after dimension recovery is carried out on a received voice semantic feature signal by utilizing a channel decoder, semantic decoding is carried out by utilizing a semantic decoder to obtain a real part and an imaginary part of a predicted pure voice signal, and finally inverse short-time Fourier transform is carried out to obtain a reconstructed voice signal. According to the invention, under the condition of saving communication resources, the quality of pure voice recovered by a remote receiving end under a low signal-to-noise ratio channel transmission condition can be obviously improved.

The invention discloses a voice quality inspection method and device, computer equipment and a storage medium, belongs to the technical field of artificial intelligence, and is applied to voice quality inspection scenes in the fields of finance, health medical care, old-age care and the like. The multi-modal feature fusion technology is introduced, the context semantic features of the text and the acoustic features of the voice are extracted, the emotion features are obtained by combining the pre-trained emotion recognition model, comprehensive understanding of the voice data from the three dimensions of semantics, acoustics and emotions is achieved, the deep fusion of the three feature vectors is carried out, and the voice recognition efficiency is improved. Compared with a traditional method which only depends on text or acoustic features, the method has the advantages that multi-modal features are realized by combining emotional features on the basis of the text or acoustic features, and information contained in voice content can be reflected more comprehensively and meticulously, so that the accuracy and practicability of voice quality inspection are improved, and the voice quality inspection efficiency is improved. And the requirements of application scenes such as intelligent customer service and voice auditing on high-quality automatic quality inspection are met.

This document relates to distributed devices teleconferencing. Some implementations can employ adaptive microphone selection based on signal characteristics such as signal-to-noise ratios or speech quality, and / or based on a microphone affinity approach. The selected microphone signals can be synchronized and mixed to generate a playback signal that is sent to a remote device. Further implementations can perform proximity-based mixing, where microphone signals received from devices in a particular room can be omitted from playback signals transmitted to other devices in the same room. These techniques can allow enhanced call quality for teleconferencing sessions where co-located users can employ their own devices to participate in a call with other users.

The invention relates to the technical field of speech recognition effect evaluation, in particular to a speech recognition evaluation method based on artificial intelligence. The method comprises the following steps: acquiring to-be-evaluated voice data, analyzing the to-be-evaluated voice data to obtain a voice fluency index, a voice clarity index and a voice accuracy index, and calculating to obtain a first voice quality index of the to-be-evaluated voice data based on the voice fluency index, the voice clarity index and the voice accuracy index; inputting the to-be-evaluated voice data into the voice content recognition model, outputting a prediction text of the to-be-evaluated voice data, and performing comparative analysis on the prediction text and the labeled text to obtain a second voice quality index of the to-be-evaluated voice data; and obtaining a voice recognition evaluation result corresponding to the to-be-evaluated voice data based on the first voice quality index and the second voice quality index. According to the invention, the evaluation efficiency and accuracy of the speech recognition effect can be improved.

The invention relates to the technical field of voice recognition, in particular to an implementation method and device of a multi-channel voiceprint recognition system, and the implementation method comprises the steps of multi-channel data acquisition and synchronization, signal preprocessing and enhancement, feature extraction and fusion, model training, real-time deployment and adaptive optimization. Compared with the problems that a traditional multichannel voiceprint recognition system depends on a fixed beam forming algorithm and an independent clock synchronization module, the synchronization error is large, manual parameter adjustment is needed for noise suppression, and generalization is poor, hardware-level clock synchronization is achieved through a PTP protocol, and the accuracy of noise suppression is improved. The method combines an end-to-end neural network to automatically learn noise distribution and a sound source space position, dynamically generates a beam forming weight, can improve the voice quality in a complex noise scene without manual intervention, remarkably reduces the interference of a synchronization error on sound source positioning, and enables the precision and stability of far-field voice enhancement to reach a new level.

The invention relates to the technical field of voice signals, in particular to a two-step mixed sound source separation and de-reverberation method, which comprises the following steps of: firstly, carrying out separation network training by taking different types of signals as training targets in various separation networks for separating attention and the like; the invention provides an improved de-reverberation method based on a time convolution network-weight prediction error, multiple improvement strategies such as taking a scale invariance signal-to-noise interference ratio as a network loss function, adopting a transposition mechanism for an input signal and a mechanism for additionally adding a residual value to a network unit are used, and finally, the de-reverberation method based on the time convolution network-weight prediction error is obtained. And cascading the separation attention network with the best separation noise reduction effect with the time convolution-transpose-residual error-WPE network. According to the invention, the problem that the separation effect of the mixed audio signal with reverberation and noise signals is not good under the actual sound field condition is solved, compared with a single one-step or two-step existing separation, de-mixing and noise reduction network, the method is significantly improved, and the quality of the separated voice can be improved for later recognition and discrimination.

The invention relates to a word prompting method and device, head-mounted equipment and a storage medium. The method comprises the steps of obtaining word prompting content; under the condition that a target object wears the head-mounted device, acquiring angle change data of the head of the target object within a preset duration; and performing page turning control on the word mentioning content according to the angle change data. According to the method, the prompt content can be displayed to the target object through the head-mounted device, and the target object wears the head-mounted device, and the head-mounted device can collect the angle change data of the head of the target object, so that page turning control can be carried out on the prompt content through the head-mounted device according to the angle change data; therefore, the speaker can be prompted secretly through the head-mounted device, the speaker is helped to improve the speech quality, and the user experience is improved.

The invention discloses a lip language-to-voice conversion method and device, a computer storage medium and a terminal, and aims to solve the problems that a lip language-to-voice conversion technology cannot be deployed on terminal equipment and voice quality cannot meet application requirements. The method is combined with a neural vocoder which reduces calculation complexity and resource requirements, system configuration requirements are reduced while system parameters are reduced, a design basis is provided for deploying a lip language-to-speech method on terminal equipment, and context-related visual feature sequences and user audio embedding vectors are fused, so that the user audio-to-speech conversion efficiency is improved, and the user audio-to-speech conversion efficiency is improved. A Mel spectrum acoustic feature sequence used for being converted into a voice waveform is obtained, and a user audio vector fused with the Mel spectrum acoustic feature sequence is obtained, so that the output voice waveform is more consistent with the real voice of a user; according to the embodiment of the invention, technical support is provided for deploying and applying the lip language-to-speech method meeting the speech quality requirement on the terminal equipment.

The embodiment of the invention belongs to the technical field of speech synthesis, is applied to a scene of converting a text into speech, and relates to a speech synthesis output method and device, equipment and a storage medium thereof. Inputting the input text into an improved text encoder, and carrying out text serialization encoding processing; obtaining a text coding sequence output by the improved text encoder, and performing synthetic speech duration prediction on the text coding sequence through a preset duration predictor to obtain predicted duration; performing voice stream synthesis on the text coding sequence according to a predicted time length to obtain a synthesized target voice stream; and inputting the target voice stream into a preset voice decoder to complete target voice output. By optimizing the encoding and decoding processing process during speech synthesis output, under the condition that the quality of the output speech is not changed, the encoding and decoding parameters are reduced, the speech synthesis processing complexity is reduced, and the synthesis processing efficiency during speech synthesis is improved.

The invention discloses a supervised learning zs-TTS generation and quality evaluation system for Tibetan language data enhancement, and belongs to the technical field of voice intelligent synthesis. The system comprises a preprocessing module, a zero sample text-to-speech synthesis module and a synthesized speech quality evaluation module. The preprocessing module processes the collected Tibetan speech data to obtain a sample; the zero sample text-to-speech synthesis module generates Tibetan speech by using a z-TTS model of supervised learning, introduces a reference encoder on the basis of the TTS model, and optimizes the composition of a loss function of a generative adversarial network; the synthetic speech quality evaluation module scores the synthetic audio in combination with a plurality of indexes, and introduces a dynamic threshold mechanism to balance the quality and number of samples. According to the system, the ability of quickly learning the style of a speaker and generating a high-quality Tibetan speech sample under the condition of extremely short-time input samples is realized, and an effective universal data enhancement scheme is provided for a speech processing task of a low-resource language.

The invention relates to a voice processing method and device, electronic equipment and a storage medium. The method comprises the following steps: extracting voice features of a damaged voice signal to obtain damaged voice features; inputting the damaged voice features into a repaired voice generation network of a voice damage repair model to obtain generated repaired voice features; dividing the repaired voice features according to a plurality of preset sub-bands to obtain sub-repaired voice features corresponding to the sub-bands; inputting the sub-repaired speech features corresponding to the sub-bands into a multi-band speech enhancement network of a speech damage repair model, and performing speech enhancement processing on the corresponding sub-repaired speech features through speech enhancement sub-networks corresponding to the sub-bands in the multi-band speech enhancement network to obtain enhanced speech features of the sub-bands; and generating a deep repair voice signal corresponding to the damaged voice signal based on the enhanced voice feature of each sub-band. According to the invention, the voice quality and intelligibility can be remarkably improved in a scene in which multiple distortions exist at the same time.

A method and system for enhancing speech in a mask, a storage medium and a program product relate to the field of speech analysis or synthesis, and the method comprises the following steps: collecting first audio samples of a plurality of target objects in a mask wearing state and second audio samples of a plurality of target objects in a mask non-wearing state; according to the first audio sample and the second audio sample, determining spectrum difference characteristics under the mask effect; performing model training based on the spectrum difference features to obtain a speech enhancement model; acquiring a real-time audio signal of a target user in the working scene, and performing noise reduction preprocessing on the real-time audio signal to obtain a preprocessed audio signal; inputting the preprocessed audio signal into a speech enhancement model to obtain an enhanced audio signal; and adjusting the enhanced audio signal according to preset user configuration to obtain an output audio signal, and outputting the target voice according to the output audio signal. According to the invention, the voice quality of sound production in the mask can be improved, and smooth communication during cooperative work of multiple persons is ensured.

The invention discloses an interphone implementation system based on voice automatic recognition, and relates to the technical field of voice recognition, and the system comprises the steps: collecting a voice signal of a user through an interphone microphone, and converting the voice signal into an analog electric signal; converting the analog electric signal into a digital audio signal by using an AAC audio coding algorithm, and carrying out coding processing; voice data packets lost in the transmission process are repaired in a data redundancy mode by using a forward error correction technology; framing the audio signal to be processed by using an overlapped sliding window method; detecting a voice area endpoint in the voice frame by using a double-threshold judgment method; training a speech recognition hidden Markov model through the speech area data and the corresponding statements in the information base; and obtaining statement information corresponding to the voice signal of the user by using a Viterbi algorithm. By arranging the audio repairing module, the endpoint detection module, the model training module and the voice recognition module, the voice quality is improved, the call distance is increased, and thus the working efficiency of a user is improved.

The invention relates to the technical field of speech enhancement, in particular to a far-field single-channel speech enhancement method based on an MFSE (Maximum Free Square Error) (Maximum Free Square Error) (Maximum Free Square Error) (Maximum Free Square Error) (Maximum Free Square Error) (Maximum Free Square Error)). The method comprises the following steps: step 1, processing a far-field voice signal to obtain a complex spectrogram of a noise voice signal; 2, inputting the compressed complex spectrogram into a feature encoder, processing the output of the complex spectrogram by N MamAttention blocks, and then sending the processed complex spectrogram into an amplitude mask decoder and a phase decoder to respectively predict a clean compressed amplitude mask and a phase spectrum; step 3, preheating and training the MamAttention model, and performing supervised confrontation training by taking the MamAttention model as a generator and the multi-resolution discriminator as a discriminator; and step 4, inputting test voice into the trained model to realize far-field single-channel voice enhancement. According to the method, the supervised adversarial training strategy and the MamAttention model are combined, so that the problems of signal attenuation, noise and reverberation interference in far-field voice are effectively solved, and the voice quality is remarkably improved in a scene that the distance of a loudspeaker exceeds 5 meters.

The invention belongs to the technical field of emotion calculation, and particularly relates to a voice emotion recognition method based on diffusion enhancement and multi-granularity feature fusion, and the method comprises the steps: converting an audio into a Mel spectrogram; the Mel spectrogram is input into a diffusion model, forward diffusion noise injection and backward diffusion noise removal are carried out step by step, and clearer audio is recovered; extracting features of the enhanced audio, then extracting frame-level features, segment-level features and utterance-level features from the audio, then performing adaptive feature selection, and performing deep fusion through weight position dependence; and inputting the fused multi-granularity speech features into a dynamic graph convolution sentiment classification model to generate a sentiment state recognition result. According to the invention, services such as multi-classification speech emotion recognition and speech quality enhancement can be provided, so that a more humanized response is provided in interaction.

The invention provides a voice recognition method and electronic equipment, and the method comprises the steps: the electronic equipment obtains source sound data in response to a wake-up operation of a user on a first application; determining a target voiceprint template based on the source sound data; the target voiceprint template is a voiceprint template with the highest matching degree with the source sound data in voiceprint templates of users corresponding to the electronic equipment; as the target voiceprint template is the voiceprint template which is most matched with the voiceprint of the user during the current speaking, the voice quality of the obtained target voice data can be improved by obtaining the target voice data according to the source voice data and the target voiceprint template, and then voice recognition is performed based on the target voice data to obtain the voice recognition result, so that the user experience is improved. The accuracy of a speech recognition result can be improved.

The invention provides a communication method for realizing a private call, which comprises the following steps that: a wireless silencer preprocesses acquired voice to improve the voice quality; establishing communication connection between the earphone and the wireless silencer; the earphone receives the voice data sent by the wireless silencer; the microphone of the earphone is turned off, and the earphone is switched to an audio output mode; the wireless silencer is worn at the mouth of a speaker so as to collect voice sent by the speaker, and first audio information obtained by converting the voice is sent to the earphone; the earphone transmits the first audio information to a call device; the earphone receives the second audio information from the call device, converts the second audio information into sound and transmits the sound to human ears; in the conversation process, the wireless silencer shields the voice sent by the speaker so as to prevent the voice from being spread to the surrounding environment, and private conversation is achieved.

The invention discloses a voice quality optimization method and device, equipment, a storage medium and a product, and relates to the technical field of communication, and the method comprises the steps: based on an RSRP sequence of each user equipment, recognizing each fast fading equipment with an RSRP fluctuation value exceeding a preset fluctuation threshold value from each user equipment through a preset recurrent neural network; determining a communication cell to which each fast fading device belongs according to the temporary identifier of each fast fading device, and determining the communication cell meeting a preset device scale condition and a fast fading device distribution condition as a fast fading cell; determining a voice problem cell from the fast fading cells according to the service quality identifier of each fast fading device in the fast fading cells; and voice quality optimization is carried out on voice problem equipment in the voice problem cell, and the voice problem equipment is fast fading equipment for executing voice services. Therefore, the influence of the fast fading phenomenon on the voice quality is reduced, and the voice quality is improved.

The invention provides a voice de-reverberation method, device and equipment and a medium, and relates to the technical field of voice signal processing, and the method comprises the steps: obtaining a reverberation signal through a microphone array, and carrying out the short-time Fourier transform of the reverberation signal, and obtaining a time-frequency domain signal; respectively processing the time-frequency domain signal by using a plurality of high-directivity beam formers to obtain a time-frequency signal; designing a dereverberation filter based on a weighted linear prediction algorithm according to the plurality of time-frequency signals; performing dereverberation filtering on the time-frequency signal output by one of the high-directivity beam formers by using a dereverberation filter to obtain an expected time-frequency domain signal after dereverberation; and performing inverse short-time Fourier transform on the expected time-frequency domain signal to obtain a reverberation-removed time-domain signal. According to the method, the performance of an existing WPE algorithm can be improved, the reverberation removing performance of the WPE algorithm in a noise environment is improved while the calculation complexity is reduced, voice reverberation can be removed, and the voice quality is improved.

This application discloses a speech enhancement method (100, 200) and device. The speech enhancement method (100, 200) includes: receiving a current audio input signal having a speech portion and a non-speech portion (201); determining a voice feature of the speech portion in the current audio input signal (202); determining a speech quality of the current audio input signal (203); evaluating whether the speech quality meets a predetermined speech quality requirement (204); and creating or updating, in response to the speech quality meeting the predetermined speech quality requirement, a reference speech feature by using the voice feature, where the reference speech feature is used for enhancing the speech portion in an audio input signal (205).

The invention provides a driving safety analysis method based on voice emotion recognition and a related device, and the method comprises the steps: obtaining a driving voice data flow collected by a target vehicle in a driving environment, carrying out the emotion feature extraction of the driving voice data flow, and generating a multi-level emotion feature set; inputting the multi-level emotion feature set into a safety risk prediction model, generating a safety behavior scoring curve of the driver in a preset time period, and generating driving intervention prompt information corresponding to risk nodes according to the risk nodes exceeding a preset threshold in the safety behavior scoring curve, and the driving intervention prompt information is sent to the vehicle-mounted terminal of the target vehicle for real-time display. According to the invention, hardware deployment cost can be reduced, interference of voice quality fluctuation on an analysis result in a complex driving environment is ensured to be effectively suppressed, and effects of improving driving safety early warning accuracy and reducing traffic accident rate are realized.

The invention relates to the technical field of voice quality detection, in particular to a voice quality detection and evaluation method based on multi-modal fusion. The method comprises the following steps: carrying out short-time Fourier transform on a multi-modal fused noisy speech signal to obtain a plurality of noisy speech spectrums, and constructing a complex spectrum matrix of the noisy speech; calculating based on the complex spectrum matrix of the noisy voice to obtain an actual value voice feature matrix corresponding to the multi-modal fused noisy voice signal, inputting the actual value voice feature matrix into the multi-modal fused voice signal reconstruction analysis model, and outputting an optimal actual value voice feature; training a deep network by taking the optimal real-value speech feature as a target to realize speech enhancement; the prior signal-to-noise ratio fusing the specific person information is calculated based on the enhanced voice signal, and quality detection and evaluation are performed on the voice signal based on the prior signal-to-noise ratio, so that the reliability and the accuracy of performing quality detection and evaluation on the multi-modal fused voice signal can be improved.

The invention relates to the technical field of intelligent equipment and man-machine interaction, and discloses a voice recognition and voice broadcast system, which comprises a text input module used for receiving text content input by a user; the voice synthesis module is used for generating a corresponding voice signal according to the text input by the user; the parameter configuration module is used for allowing a user to customize voice parameters including timbre, speed, volume, tone and brightness; the file generation module is used for converting the synthesized voice into a downloadable audio format file; and the downloading management module is used for providing storage and downloading of the voice file. A user can adjust voice parameters such as timbre, speech speed, volume, tone, brightness and the like, audio files in various formats are generated, and management of local storage and cloud storage is supported. Compared with the prior art, more flexible voice customization, higher voice quality and higher equipment compatibility are provided, and the user experience is remarkably improved.

The invention relates to the technical field of intelligent voice, can be applied to the fields of finance and medical treatment, and discloses an unsupervised learning voice quality evaluation method, device, equipment and medium, and the method comprises the steps: extracting multi-dimensional voice features in a voice signal; constructing a voice feature model based on an unsupervised learning auto-encoder structure, learning quality representation of voice features, comprehensively evaluating the quality of the multi-dimensional voice features according to a learning result, and generating a quality score of the voice features; comparing the clear voice data, the distorted voice data and the unlabeled voice data, and optimizing the voice feature model according to a comparison result; and training the optimized voice feature model, obtaining the difference between the unlabeled voice data and the voice features in the optimized voice feature model, carrying out fine tuning on the trained voice feature model by taking the difference as a fine tuning demand, and outputting a new quality score of the voice features.

The invention relates to the technical field of multi-speaker speech synthesis, and particularly provides a multi-speaker speech synthesis method and device, a storage medium and computer equipment. According to the method, customized structure expansion, engine transformation and optimization are carried out on the basis of an open source reasoning framework vLLM, so that the target vLLM framework obtained through transformation can face a speech synthesis scene, and efficient reasoning of a speech synthesis model is supported. Speech synthesis reasoning is realized by using the target vLLM framework and the LLM-based target speech synthesis model, so that speech synthesis can be accelerated under the condition of keeping speech quality.

The embodiment of the invention discloses a voice quality inspection method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring voice data; performing emotion recognition on the voice data based on the acoustic features of the voice data to obtain an emotion recognition result; performing voice recognition on the voice data to obtain voice text data; and constructing a first prompt word according to the emotion recognition result and the voice text data, and processing the first prompt word by using a pre-trained text quality inspection model to obtain a voice quality inspection result. According to the embodiment of the invention, the quality inspection accuracy of the voice data can be improved.

The invention relates to a zero-sample speech synthesis method and device, computer equipment, a storage medium and a program product, and the method comprises the steps: obtaining a target coding feature according to a reference speech and a target text; inputting the target coding features into a stream matching model to obtain a conditional velocity field and an unconditional velocity field; inputting the target coding feature into a prior model to obtain a prior speech feature; obtaining a prior generation flow field according to the prior voice features and the standard Gaussian noise; calculating a KL divergence value between the priori generated flow field and a preset real generated flow field, and taking a moment when the KL divergence value is smaller than or equal to a preset KL divergence threshold as an initial moment of the priori generated flow field; fusing the conditional velocity field and the unconditional velocity field to obtain a fused velocity field; and inputting the fusion velocity field from the initial moment to the target moment and the priori generated flow field into an ordinary differential equation solver to obtain the target speech features, thereby improving the speech quality of the synthesized speech.