309 results about "Speech quality" patented technology

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of medical health, financial science and technology, cultural transmission and the like, and discloses a voice generation method based on multi-modal fusion, which comprises the following steps: collecting audio data to extract timbre features, and training a field feature timbre generation model; analyzing text semantic recognition emotion information, adjusting speech synthesis parameters, combining personalized information to construct a parameter mapping table, fusing to generate a synthesis control parameter sequence, aligning the synthesis control parameter sequence with character labels, visual elements and background music data, driving a domain feature timbre generation model, and generating synthesis data of synchronous speech, text, vision and music. Domain timbres are generated through timbre feature training, speech expression is optimized by combining semantic analysis and emotion recognition, user requirements are matched based on personalized information, and time alignment is performed by fusing text, vision and music data, so that the synthesized speech has domain features, emotion adaptability and personalization, and the speech quality is improved. And the voice immersion and the information transmission capability are improved.

The invention discloses a video translation method and system based on artificial intelligence. The method relates to the technical field of video translation and comprises the following steps of original sound track extraction, target AI speaker adaptation, AI dubbing generation and mouth shape synchronization and video synthesis. According to the method, independent audio and video streams are obtained by adopting an audio and video separation technology, and multiple original sound tracks are extracted through a voice separation model; matching or generating an adaptive target AI speaker module in a preset tone library; converting the original language voice into a text, translating the text into a target language text, and synthesizing an AI dubbing audio track in combination with a target AI speaker module; and finally, the independent video stream and the multi-AI dubbing audio track are input into the mouth shape synchronization model to output a translated video, so that the timbre fitting degree, the voice quality and the voice consistency of the same speaker of AI dubbing are improved, and meanwhile, the resource utilization rate of video translation and the processing efficiency under batch tasks are improved. The problem that in the prior art, video translation is low in quality and efficiency is solved.

The invention discloses a voice quality inspection method and device, computer equipment and a storage medium, belongs to the technical field of artificial intelligence, and is applied to voice quality inspection scenes in the fields of finance, health medical care, old-age care and the like. The multi-modal feature fusion technology is introduced, the context semantic features of the text and the acoustic features of the voice are extracted, the emotion features are obtained by combining the pre-trained emotion recognition model, comprehensive understanding of the voice data from the three dimensions of semantics, acoustics and emotions is achieved, the deep fusion of the three feature vectors is carried out, and the voice recognition efficiency is improved. Compared with a traditional method which only depends on text or acoustic features, the method has the advantages that multi-modal features are realized by combining emotional features on the basis of the text or acoustic features, and information contained in voice content can be reflected more comprehensively and meticulously, so that the accuracy and practicability of voice quality inspection are improved, and the voice quality inspection efficiency is improved. And the requirements of application scenes such as intelligent customer service and voice auditing on high-quality automatic quality inspection are met.

The invention relates to the technical field of voice recognition, in particular to an implementation method and device of a multi-channel voiceprint recognition system, and the implementation method comprises the steps of multi-channel data acquisition and synchronization, signal preprocessing and enhancement, feature extraction and fusion, model training, real-time deployment and adaptive optimization. Compared with the problems that a traditional multichannel voiceprint recognition system depends on a fixed beam forming algorithm and an independent clock synchronization module, the synchronization error is large, manual parameter adjustment is needed for noise suppression, and generalization is poor, hardware-level clock synchronization is achieved through a PTP protocol, and the accuracy of noise suppression is improved. The method combines an end-to-end neural network to automatically learn noise distribution and a sound source space position, dynamically generates a beam forming weight, can improve the voice quality in a complex noise scene without manual intervention, remarkably reduces the interference of a synchronization error on sound source positioning, and enables the precision and stability of far-field voice enhancement to reach a new level.

The invention relates to the technical field of voice signals, in particular to a two-step mixed sound source separation and de-reverberation method, which comprises the following steps of: firstly, carrying out separation network training by taking different types of signals as training targets in various separation networks for separating attention and the like; the invention provides an improved de-reverberation method based on a time convolution network-weight prediction error, multiple improvement strategies such as taking a scale invariance signal-to-noise interference ratio as a network loss function, adopting a transposition mechanism for an input signal and a mechanism for additionally adding a residual value to a network unit are used, and finally, the de-reverberation method based on the time convolution network-weight prediction error is obtained. And cascading the separation attention network with the best separation noise reduction effect with the time convolution-transpose-residual error-WPE network. According to the invention, the problem that the separation effect of the mixed audio signal with reverberation and noise signals is not good under the actual sound field condition is solved, compared with a single one-step or two-step existing separation, de-mixing and noise reduction network, the method is significantly improved, and the quality of the separated voice can be improved for later recognition and discrimination.

The invention discloses a lip language-to-voice conversion method and device, a computer storage medium and a terminal, and aims to solve the problems that a lip language-to-voice conversion technology cannot be deployed on terminal equipment and voice quality cannot meet application requirements. The method is combined with a neural vocoder which reduces calculation complexity and resource requirements, system configuration requirements are reduced while system parameters are reduced, a design basis is provided for deploying a lip language-to-speech method on terminal equipment, and context-related visual feature sequences and user audio embedding vectors are fused, so that the user audio-to-speech conversion efficiency is improved, and the user audio-to-speech conversion efficiency is improved. A Mel spectrum acoustic feature sequence used for being converted into a voice waveform is obtained, and a user audio vector fused with the Mel spectrum acoustic feature sequence is obtained, so that the output voice waveform is more consistent with the real voice of a user; according to the embodiment of the invention, technical support is provided for deploying and applying the lip language-to-speech method meeting the speech quality requirement on the terminal equipment.

The embodiment of the invention belongs to the technical field of speech synthesis, is applied to a scene of converting a text into speech, and relates to a speech synthesis output method and device, equipment and a storage medium thereof. Inputting the input text into an improved text encoder, and carrying out text serialization encoding processing; obtaining a text coding sequence output by the improved text encoder, and performing synthetic speech duration prediction on the text coding sequence through a preset duration predictor to obtain predicted duration; performing voice stream synthesis on the text coding sequence according to a predicted time length to obtain a synthesized target voice stream; and inputting the target voice stream into a preset voice decoder to complete target voice output. By optimizing the encoding and decoding processing process during speech synthesis output, under the condition that the quality of the output speech is not changed, the encoding and decoding parameters are reduced, the speech synthesis processing complexity is reduced, and the synthesis processing efficiency during speech synthesis is improved.

The invention relates to the technical field of speech enhancement, in particular to a far-field single-channel speech enhancement method based on an MFSE (Maximum Free Square Error) (Maximum Free Square Error) (Maximum Free Square Error) (Maximum Free Square Error) (Maximum Free Square Error) (Maximum Free Square Error)). The method comprises the following steps: step 1, processing a far-field voice signal to obtain a complex spectrogram of a noise voice signal; 2, inputting the compressed complex spectrogram into a feature encoder, processing the output of the complex spectrogram by N MamAttention blocks, and then sending the processed complex spectrogram into an amplitude mask decoder and a phase decoder to respectively predict a clean compressed amplitude mask and a phase spectrum; step 3, preheating and training the MamAttention model, and performing supervised confrontation training by taking the MamAttention model as a generator and the multi-resolution discriminator as a discriminator; and step 4, inputting test voice into the trained model to realize far-field single-channel voice enhancement. According to the method, the supervised adversarial training strategy and the MamAttention model are combined, so that the problems of signal attenuation, noise and reverberation interference in far-field voice are effectively solved, and the voice quality is remarkably improved in a scene that the distance of a loudspeaker exceeds 5 meters.

The invention provides a voice recognition method and electronic equipment, and the method comprises the steps: the electronic equipment obtains source sound data in response to a wake-up operation of a user on a first application; determining a target voiceprint template based on the source sound data; the target voiceprint template is a voiceprint template with the highest matching degree with the source sound data in voiceprint templates of users corresponding to the electronic equipment; as the target voiceprint template is the voiceprint template which is most matched with the voiceprint of the user during the current speaking, the voice quality of the obtained target voice data can be improved by obtaining the target voice data according to the source voice data and the target voiceprint template, and then voice recognition is performed based on the target voice data to obtain the voice recognition result, so that the user experience is improved. The accuracy of a speech recognition result can be improved.

The invention provides a communication method for realizing a private call, which comprises the following steps that: a wireless silencer preprocesses acquired voice to improve the voice quality; establishing communication connection between the earphone and the wireless silencer; the earphone receives the voice data sent by the wireless silencer; the microphone of the earphone is turned off, and the earphone is switched to an audio output mode; the wireless silencer is worn at the mouth of a speaker so as to collect voice sent by the speaker, and first audio information obtained by converting the voice is sent to the earphone; the earphone transmits the first audio information to a call device; the earphone receives the second audio information from the call device, converts the second audio information into sound and transmits the sound to human ears; in the conversation process, the wireless silencer shields the voice sent by the speaker so as to prevent the voice from being spread to the surrounding environment, and private conversation is achieved.

The invention discloses a voice quality optimization method and device, equipment, a storage medium and a product, and relates to the technical field of communication, and the method comprises the steps: based on an RSRP sequence of each user equipment, recognizing each fast fading equipment with an RSRP fluctuation value exceeding a preset fluctuation threshold value from each user equipment through a preset recurrent neural network; determining a communication cell to which each fast fading device belongs according to the temporary identifier of each fast fading device, and determining the communication cell meeting a preset device scale condition and a fast fading device distribution condition as a fast fading cell; determining a voice problem cell from the fast fading cells according to the service quality identifier of each fast fading device in the fast fading cells; and voice quality optimization is carried out on voice problem equipment in the voice problem cell, and the voice problem equipment is fast fading equipment for executing voice services. Therefore, the influence of the fast fading phenomenon on the voice quality is reduced, and the voice quality is improved.

The invention provides a voice de-reverberation method, device and equipment and a medium, and relates to the technical field of voice signal processing, and the method comprises the steps: obtaining a reverberation signal through a microphone array, and carrying out the short-time Fourier transform of the reverberation signal, and obtaining a time-frequency domain signal; respectively processing the time-frequency domain signal by using a plurality of high-directivity beam formers to obtain a time-frequency signal; designing a dereverberation filter based on a weighted linear prediction algorithm according to the plurality of time-frequency signals; performing dereverberation filtering on the time-frequency signal output by one of the high-directivity beam formers by using a dereverberation filter to obtain an expected time-frequency domain signal after dereverberation; and performing inverse short-time Fourier transform on the expected time-frequency domain signal to obtain a reverberation-removed time-domain signal. According to the method, the performance of an existing WPE algorithm can be improved, the reverberation removing performance of the WPE algorithm in a noise environment is improved while the calculation complexity is reduced, voice reverberation can be removed, and the voice quality is improved.

The invention provides a driving safety analysis method based on voice emotion recognition and a related device, and the method comprises the steps: obtaining a driving voice data flow collected by a target vehicle in a driving environment, carrying out the emotion feature extraction of the driving voice data flow, and generating a multi-level emotion feature set; inputting the multi-level emotion feature set into a safety risk prediction model, generating a safety behavior scoring curve of the driver in a preset time period, and generating driving intervention prompt information corresponding to risk nodes according to the risk nodes exceeding a preset threshold in the safety behavior scoring curve, and the driving intervention prompt information is sent to the vehicle-mounted terminal of the target vehicle for real-time display. According to the invention, hardware deployment cost can be reduced, interference of voice quality fluctuation on an analysis result in a complex driving environment is ensured to be effectively suppressed, and effects of improving driving safety early warning accuracy and reducing traffic accident rate are realized.

The invention relates to the technical field of voice quality detection, in particular to a voice quality detection and evaluation method based on multi-modal fusion. The method comprises the following steps: carrying out short-time Fourier transform on a multi-modal fused noisy speech signal to obtain a plurality of noisy speech spectrums, and constructing a complex spectrum matrix of the noisy speech; calculating based on the complex spectrum matrix of the noisy voice to obtain an actual value voice feature matrix corresponding to the multi-modal fused noisy voice signal, inputting the actual value voice feature matrix into the multi-modal fused voice signal reconstruction analysis model, and outputting an optimal actual value voice feature; training a deep network by taking the optimal real-value speech feature as a target to realize speech enhancement; the prior signal-to-noise ratio fusing the specific person information is calculated based on the enhanced voice signal, and quality detection and evaluation are performed on the voice signal based on the prior signal-to-noise ratio, so that the reliability and the accuracy of performing quality detection and evaluation on the multi-modal fused voice signal can be improved.

The invention relates to the technical field of intelligent equipment and man-machine interaction, and discloses a voice recognition and voice broadcast system, which comprises a text input module used for receiving text content input by a user; the voice synthesis module is used for generating a corresponding voice signal according to the text input by the user; the parameter configuration module is used for allowing a user to customize voice parameters including timbre, speed, volume, tone and brightness; the file generation module is used for converting the synthesized voice into a downloadable audio format file; and the downloading management module is used for providing storage and downloading of the voice file. A user can adjust voice parameters such as timbre, speech speed, volume, tone, brightness and the like, audio files in various formats are generated, and management of local storage and cloud storage is supported. Compared with the prior art, more flexible voice customization, higher voice quality and higher equipment compatibility are provided, and the user experience is remarkably improved.

The invention relates to the technical field of intelligent voice, can be applied to the fields of finance and medical treatment, and discloses an unsupervised learning voice quality evaluation method, device, equipment and medium, and the method comprises the steps: extracting multi-dimensional voice features in a voice signal; constructing a voice feature model based on an unsupervised learning auto-encoder structure, learning quality representation of voice features, comprehensively evaluating the quality of the multi-dimensional voice features according to a learning result, and generating a quality score of the voice features; comparing the clear voice data, the distorted voice data and the unlabeled voice data, and optimizing the voice feature model according to a comparison result; and training the optimized voice feature model, obtaining the difference between the unlabeled voice data and the voice features in the optimized voice feature model, carrying out fine tuning on the trained voice feature model by taking the difference as a fine tuning demand, and outputting a new quality score of the voice features.

The invention relates to the technical field of multi-speaker speech synthesis, and particularly provides a multi-speaker speech synthesis method and device, a storage medium and computer equipment. According to the method, customized structure expansion, engine transformation and optimization are carried out on the basis of an open source reasoning framework vLLM, so that the target vLLM framework obtained through transformation can face a speech synthesis scene, and efficient reasoning of a speech synthesis model is supported. Speech synthesis reasoning is realized by using the target vLLM framework and the LLM-based target speech synthesis model, so that speech synthesis can be accelerated under the condition of keeping speech quality.

The embodiment of the invention discloses a voice quality inspection method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring voice data; performing emotion recognition on the voice data based on the acoustic features of the voice data to obtain an emotion recognition result; performing voice recognition on the voice data to obtain voice text data; and constructing a first prompt word according to the emotion recognition result and the voice text data, and processing the first prompt word by using a pre-trained text quality inspection model to obtain a voice quality inspection result. According to the embodiment of the invention, the quality inspection accuracy of the voice data can be improved.

The invention relates to a zero-sample speech synthesis method and device, computer equipment, a storage medium and a program product, and the method comprises the steps: obtaining a target coding feature according to a reference speech and a target text; inputting the target coding features into a stream matching model to obtain a conditional velocity field and an unconditional velocity field; inputting the target coding feature into a prior model to obtain a prior speech feature; obtaining a prior generation flow field according to the prior voice features and the standard Gaussian noise; calculating a KL divergence value between the priori generated flow field and a preset real generated flow field, and taking a moment when the KL divergence value is smaller than or equal to a preset KL divergence threshold as an initial moment of the priori generated flow field; fusing the conditional velocity field and the unconditional velocity field to obtain a fused velocity field; and inputting the fusion velocity field from the initial moment to the target moment and the priori generated flow field into an ordinary differential equation solver to obtain the target speech features, thereby improving the speech quality of the synthesized speech.

The invention discloses a voice wake-up interaction method and system based on a microphone array, and the method comprises the steps: carrying out VAD processing, so as to judge whether a target audio segment has voice or not; voice and noise source directions are obtained; selecting beam parameters of voice and noise in combination with a pre-designed fixed beam; whether GSC module processing is carried out or not is selected according to the difference between the beam parameters of the voice and the noise, so that an enhanced audio signal is obtained, a wake-up task is carried out, and a final voice wake-up result is obtained; and according to whether the wake-up is successful, determining whether to lock the voice beam direction in the current interaction stage for enhancement, thereby preventing interference of voice in other directions on subsequent interaction tasks. According to the voice enhancement mode based on the microphone array, sound source positioning can be realized, interference in a non-target direction can be suppressed, the voice quality in the target direction can be improved, the wake-up success rate can be effectively improved when the voice enhancement mode is applied to voice wake-up, and then the experience of back-end voice interaction is improved.

The invention relates to the technical field of voice communication, and discloses a frequency domain noise reduction and reduction method and system for double-group human voice synchronous acquisition, and the method comprises the steps: obtaining a synchronous signal through double groups of acquisition points, constructing a double-source frequency domain coordinate system, and mapping the coordinate system into an energy distribution point; calculating a spatial relationship to identify an emotional distortion state, and dividing four quadrants to generate a distortion type mark; dividing a frequency domain signal and executing targeted reconstruction and correction, and finally dividing a frequency band enhanced energy remarkably-reduced part; according to the method, the voice composite distortion under the extreme emotion can be accurately processed, and reliable voice quality guarantee is provided for accurate transmission of key information in an emergency communication scene.

The invention provides a speech enhancement method based on BGRU and reference noise, which comprises the following network structures: during reasoning, receiving two paths of input: reference noise and noisy speech, inputting the two paths of noise into an enhancement network module through a short-time Fourier transform encoder respectively to obtain noise estimation and speech estimation, performing inverse short-time Fourier transform on the voice estimation to obtain voice after noise reduction; during training, four paths of input including reference noise, noisy voice, noise truth value and voice truth value are received, the four paths of noise respectively pass through the short-time Fourier transform encoder, and the encoded reference noise and noisy voice are calculated through the enhancement network module to obtain noise estimation and voice estimation; and the encoded voice truth value, noise truth value, noise estimation and voice estimation are all sent to a 3CL loss module, loss is calculated and fed back to the enhanced network module, parameters are adjusted, and the performance of the enhanced network module is improved. According to the invention, the voice quality and intelligibility can be improved under the condition of extremely low signal-to-noise ratio.

The embodiment of the invention discloses a reasoning acceleration method for a stream matching TTS model, electronic equipment and a storage medium, and the method comprises the steps: carrying out the reasoning of input data through a TTS model, and recording a sampling track; performing qualitative analysis on the recorded sampling track to form a sampling strategy after preliminary pruning; reasoning the input data again by the TTS model by using the sampling strategy after the preliminary pruning; evaluating the quality of the generated voice in a mode of combining objective indexes and subjective evaluation, and judging whether the quality of the generated voice meets a preset standard or not; when the sampling step number reaches the standard, judging whether the sampling step number can be further reduced on the current basis or not; if the sampling step number can be further reduced, the previous reasoning and analysis processes are repeated after corresponding steps are reduced; when the generation quality does not reach the standard, returning part of the steps which are pruned before, and adjusting a sampling strategy; and when the voice quality reaches the standard and the step number cannot be further reduced, storing a sampling strategy for the TTS model.

The invention discloses a Chinese-English bilingual text-to-speech conversion method and system based on NPU, and belongs to the technical field of speech processing, and the method comprises the steps: carrying out the word segmentation processing and phoneme conversion of a Chinese-English mixed text based on the language type of each segment in the Chinese-English mixed text, and obtaining a text input sequence; performing vector combination on the phoneme ID sequence and the language ID sequence, and extracting text semantic features from a vector combination result to obtain text hidden variables; according to the text hidden variable, predicting the duration of each phoneme and a prior distribution parameter in an acoustic potential feature space; performing multi-level transformation on the phoneme alignment result and the prior distribution parameter to obtain a bilingual acoustic feature sequence; and converting the bilingual acoustic feature sequence into a corresponding voice waveform. Therefore, by implementing the method, the device and the system, the problems that the Chinese-English bilingual text occupies more resources in the voice conversion process and the output voice quality is lower in the prior art can be solved.

The invention discloses an intelligent speech synthesis system based on an electronic artificial throat audio input signal. The intelligent speech synthesis system is characterized by comprising a signal processing module, a speech recognition module, a speech synthesis module and a user interface module. Compared with the prior art, the method has the advantages that STFT is used for converting the electronic throat signals into frequency domain features, meanwhile, a Conformer model is adopted for semantic extraction, natural voice is generated in combination with FastSpeech2 and Hi Fi-GAN, and emotion expression and voice quality are improved.

The present application relates to the field of speech synthesis, and particularly relates to a speech synthesis style migration method and device, computer equipment and a storage medium. The method comprises the following steps: inputting the text data to be converted into a speech synthesis encoder to obtain text encoding; obtaining multi-level speech style representation of the speech to be migrated; inputting the text data to be converted and the multi-level speech style representation into a style predictor to obtain a predicted speech style; performing multi-style layer regularization on the text encoding and the predicted speech style to obtain style-regularized text encoding; inputting the style-regularized text encoding into a pitch predictor to obtain rhythm change data; inputting the rhythm change data and the style-regularized text encoding into a speech synthesis decoder to obtain a mel-frequency spectrum diagram and generate target speech. The present application considers rhythm information and the style of words and phonemes, and the overall rhythm style of the speaker, thereby improving the speech quality of the target speech.

The invention belongs to the field of interphones, and provides a mobile phone auxiliary type intelligent interphone and an implementation method thereof, and the mobile phone auxiliary type intelligent interphone comprises a mobile phone end assembly which comprises a central processing unit, an audio processing chip, a first connection module, a loudspeaker and a microphone; the interphone end assembly comprises a main control chip, a wireless communication module, a second connection module and a power management module; a radio frequency transceiver chip is arranged in the wireless communication module, data transmission is carried out between the second connection module and the first connection module, and the audio processing chip adopts a CNN noise reduction model to process digital audio signals; according to the method, the digital audio signals are processed through the CNN noise reduction model, various complex background noises can be accurately identified and removed, clear voice signals can be restored, the voice definition and intelligibility are greatly improved, it is ensured that both communication parties can accurately transmit information in a noisy environment, and the user experience is improved. The problem of poor voice quality caused by noise interference in a traditional matching mode is effectively solved.

The present invention discloses a Mongolian automatic speech quality assessment method based on hierarchical transfer learning, which includes the following steps: pre-training an English speech self-supervised model and an English speech quality assessment model to obtain a trained English speech self-supervised model and a trained English speech quality assessment model; performing transfer learning on the trained English speech self-supervised model and the trained English speech quality assessment model to obtain a trained self-supervised model and a trained speech quality assessment model; using the trained speech self-supervised model and the BERT model to extract the feature vectors of Mongolian speech and the text features in the corresponding text; fusing the speech features and the text features into sentence-level semantic features f, and sending f into the trained speech quality assessment model to obtain the MOS score z corresponding to the speech signal, completing the automatic assessment of Mongolian speech quality, pioneering an automatic assessment method for Mongolian speech quality and filling the gap in this field.

The present disclosure relates to an audio signal processing method, apparatus, device and storage medium. The present disclosure obtains a first feature map by encoding a first complex spectrum map corresponding to the original audio signal, and obtains a second feature map by processing the time series and frequency series corresponding to the first feature map respectively. Based on the second feature map, complex ratio masking and complex spectrum mapping are simultaneously learned. Thus, the amplitude spectrum and phase of the original audio signal are enhanced simultaneously by combining masking prediction and spectrum prediction, thereby improving the speech enhancement effect. Speech enhancement can greatly improve speech quality, solve noise interference problems, and improve speech recognition effects.

The invention provides an audio processing method and device, a server and a storage medium. The method comprises the following steps: receiving a to-be-adjusted original audio and a corresponding line; converting the lines into a corresponding text sequence; performing alignment operation on the original audio and the text sequence to obtain an audio-text corresponding relation; receiving a target text; determining a to-be-replaced first text from the text sequence according to the target text and frame information of the first text in the audio in the corresponding relation; replacing the first text with the target text by using the frame information to obtain an updated text sequence; and generating a new audio by using the updated text sequence. Therefore, the problems that the edited voice is stiff, the complex audio effect is poor, and the quality is possibly poor if the voice is modified too much can be solved, and the quality of the generated voice can be better controlled by controlling the durations of different granularities.