97 results about "Speech code" patented technology

The invention relates to the technical field of speech synthesis, can be applied to business scenes such as financial science and technology, medical health and the like, and discloses a speech generation method, device, equipment and medium. And inputting a text speech language model to generate an intermediate code in combination with a speech code extracted based on a codebook generation mode, extracting a speaker feature vector in the prompt speech, decoding the intermediate code and the speaker feature vector by a generative adversarial decoder, and outputting a target speech. According to the method, fine-grained text representation is established by fusing character semantics and pinyin pronunciation information, and voice cloning is completed through unified voice language modeling and an adversarial generation mechanism in combination with codebook-driven acoustic coding and speaker personality characteristics, so that the naturalness, similarity and pronunciation accuracy of generated voices are improved.

The embodiment of the invention discloses a discrete voice coding method, electronic equipment and a storage medium, and the method comprises the steps: constructing a voice input and frequency spectrum output VAE, enabling the input voice to have speaker tone disturbance, and enabling the VAE structure to be used for multi-task learning; vector quantization is carried out on the VAE, and a VQ-VAE is initialized, so that a quantization layer is added to continue to reduce an information bottleneck; and additionally training a vocoder, and carrying out waveform reconstruction by starting from the code obtained in the VQ-VAE.

The invention provides a training method of a speech recognition model and a speech recognition method, and the training method comprises the steps: extracting the speech features of label sample speech based on a speech coding network, decoding the speech features based on a decoding network, and obtaining the alignment information of the label sample speech; the coding module is used for extracting text features of label texts of the label sample voice based on the text coding network, performing feature alignment on the text features based on the alignment information to obtain aligned text features, and sharing parts of the voice coding network and the text coding network; performing parameter iteration on the voice coding network, the text coding network and the decoding network based on the voice features and the aligned text features; and constructing a speech recognition model based on the speech coding network and the speech decoding network which are completed by parameter iteration. According to the method provided by the invention, the generalization ability of the speech recognition model can be effectively improved through the application of the multi-modal training data and the coding module shared by the speech coding network and the text coding network.

The invention discloses a fine emotion control TTS method and device based on a large model, equipment and a medium, relates to the technical field of artificial intelligence, and can generate a voice service with fine emotion expression and improve the user experience in the high-sensitivity emotion interaction fields of banks, financial customer service, insurance consultation, medical hospital guide and the like. Acquiring an emotional feature vector of the input text based on a preset large language model; encoding the emotion feature vector to obtain a voice encoding vector; determining an emotion curve vector of the input text based on a pre-trained neural network model; and generating target voice corresponding to the input text based on the voice coding vector and the emotion curve vector. According to the method, emotion feature vectors are extracted through a large language model, a dynamic emotion curve is generated in combination with a neural network, speech synthesis is cooperatively driven, emotion fineness and continuity are remarkably improved, and the problems of traditional TTS emotion expression roughening and fragmentation are solved.

The present disclosure provides a pitch-based speech conversion model training method and a speech conversion system, wherein an audio feature code is output by a priori encoder, and a pitch feature is extracted by a pitch extraction module. A linear spectrum corresponding to the reference speech is input into the posteriori encoder to obtain an audio latent variable. In addition, the audio feature code, a speech concatenation feature obtained by concatenation of the audio feature code and the pitch feature, and the audio latent variable are input into a temporal alignment module to obtain a converted speech code, and the converted speech code is decoded by a decoder to obtain a converted speech. The training loss of the converted speech is then calculated to determine the degree of convergence of the speech conversion model.

The invention discloses a layered voice coding and decoding method based on MDCT coefficient residual vector quantization, which comprises the following steps: preprocessing voice data for training, and training to generate a multi-level quantization codebook of MDCT; the method comprises the following steps of: enhancing a high-frequency component of voice to be coded through a pre-emphasis filter, windowing a pre-emphasis voice signal, generating an MDCT coefficient based on an FFT (Fast Fourier Transform) algorithm, determining a bit allocation mode according to a code rate to generate a coding control bit, compressing a codebook index and the coding control bit through Huffman coding, and transmitting a code stream to a channel; the method comprises the steps of reading a code stream in a channel, obtaining a coding control bit and an MDCT codebook index through Huffman decoding, determining a voice sampling rate and a bit distribution mode through the coding control bit, obtaining an MDCT vector based on MDCT codebook index retrieval, and restoring an original voice signal through de-emphasis filtering. Through the design of MDCT layered coding, the coder can dynamically adjust coding bit distribution according to the code rate condition, so that the method adapts to the actual communication environment and deals with the sudden network condition.

The invention relates to a speech recognition method, device and equipment based on an end-to-end cross-language large model. The method comprises the following steps: constructing a cross-language large model; obtaining a training data set formed by multiple groups of voice text label pairs, and inputting the training data set into the cross-language large model for end-to-end training until a trained cross-language large model is obtained; a to-be-recognized voice signal is input into the trained cross-language model, and filter feature and wav2vec2 feature extraction and splicing, text embedded feature representation, dimension alignment, splicing and decoding translation are carried out through a voice coding module, a text coding module, a feature alignment module and a decoding module in the model in sequence. And outputting and obtaining the speech recognition text of the target language. By adopting the method, the accuracy and robustness of speech recognition can be improved, and cross-language speech recognition is realized.

The invention relates to the field of artificial intelligence, and can improve the fluency and naturalness of multilingual speech synthesis in the aspect of speech code conversion. Relates to a speech synthesis method and device, equipment and a medium. The method comprises the following steps: acquiring initial feature vectors respectively corresponding to at least two types of initial speech data; calling the trained speech synthesis model; determining a target language expert unit corresponding to each type of initial voice data, and inputting the initial feature vector corresponding to each type of initial voice data into the corresponding target language expert unit for feature extraction to obtain first feature information corresponding to each type of initial voice data; inputting the initial feature vectors corresponding to all the initial voice data into a language-independent expert unit for feature extraction to obtain second feature information; determining target feature information according to the second feature information and the first feature information corresponding to each type of initial voice data; and performing voice synthesis on the target feature information to obtain target voice data.

The invention discloses a barrier speech recognition and reconstruction method based on a speech data retrieval enhancement technology. The method comprises the following steps: segmenting a to-be-recognized obstacle speech into a plurality of segments; for each segment, obtaining a plurality of similar voice segments and corresponding text contents by retrieving a barrier voice database; respectively coding the similar voice segments and the to-be-recognized obstacle voice to obtain similar voice codes and to-be-recognized obstacle voice codes; respectively converting the similar speech codes and the to-be-recognized obstacle speech codes into first codes and second codes understood by a large language model; the first code and the second code are input into a large language model, and text information corresponding to the to-be-recognized obstacle voice is obtained by using set cue words, and the cue words comprise historical dialogue content of the to-be-recognized obstacle voice and text content corresponding to similar voice segments; and performing voice reconstruction on the recognized text information. According to the invention, the accuracy and generalization ability of obstacle speech recognition can be improved.

The embodiment of the invention relates to a method and device for training a voice coding model, equipment and a readable medium. The method comprises: processing a speech feature representation of a speech sample using a first speech coding model to generate a set of discrete features; generating label information based on the set of discrete features, the label information comprising a set of labels, the set of labels indicating clustering centers corresponding to the corresponding discrete features; processing an intermediate feature representation generated based on the voice feature representation by using a second voice coding model to generate probability information corresponding to the tag information; the training loss is determined based on the label information, the probability information and the weight information, and the weight information is determined based on the distance from each discrete feature to the corresponding clustering center; and adjusting parameters of the second speech coding model based on the training loss. Therefore, the data volume required by training can be reduced, the stability of the unsupervised model training process can be improved, the training effect of the model is improved, and the model capability is further improved.

The invention provides an ultra-low bit rate voice coding and decoding system based on text semantic information fidelity, and relates to the technical field of voice coding and decoding. The system comprises the following steps: performing voice feature extraction and text feature extraction on original voice through a multi-modal text-voice combined encoder to obtain voice features and text features, embedding the text features into the voice features to obtain text-voice features of the original voice, and performing voice feature extraction and text feature extraction on the original voice through a multi-modal text-voice combined encoder; sending the text-speech features of the original speech to a receiving end in a semantic communication system; and the receiving end inputs the received text-speech features into a text semantic fidelity decoder based on an attention mechanism to obtain reconstructed speech features and reconstructed text features, decodes the reconstructed speech features layer by layer, and performs attention calculation on the reconstructed speech features and the reconstructed text features to obtain decoded speech. By means of the voice coding and decoding technology, good voice perception quality can still be kept under the ultra-low code rate, and voice intelligibility is guaranteed.

The application provides a speech recognition text scoring method and device, electronic equipment and storage medium. The speech recognition text scoring method obtains standard audio features of target speech based on a speech coding process in a speech recognition process of the target speech from the perspective of speech recognition, thereby providing a reference standard for scoring of the recognized text. From the perspective of scoring of the recognized text, audio distribution features corresponding to the first recognized text are obtained based on the first recognized text of the target speech. The first recognized text is scored in combination with the audio distribution features and the standard audio features. The method considers the correlation between the speech recognition text and the audio during scoring of the recognized text, thereby achieving scoring of the speech recognition text.

The invention provides a speech recognition method and device, electronic equipment and a storage medium, and relates to the technical field of natural language processing, an adopted target automatic speech recognition model generates context semantic representation through a current text prefix sequence, and combines the context semantic representation with the current text prefix sequence and acoustic features to obtain a speech recognition result. A target text sequence is obtained through step-by-step prediction in an autoregression mode. Context semantic representation is introduced into the target automatic speech recognition model, the language switching moment can be accurately judged when the input speech signal is speech code conversion speech, speech recognition is carried out in time according to a new language during language switching, the recognition precision and robustness of a language switching boundary can be effectively improved, and the speech recognition efficiency is improved. And the speech code conversion speech recognition effect is improved. Moreover, context semantic representation is introduced during prediction, the challenge of ambiguity or ambiguity of acoustic signals can be overcome, the accuracy of the target text sequence is improved, and errors caused by untimely language model switching are reduced.

The present application belongs to the technical field of communications. Disclosed are a call establishment method, an apparatus and a network side device. The call establishment method in the embodiments of the present application comprises: a first core network element receives a first call establishment request message sent by a first terminal; and the first core network element sends a second call establishment request message to a second terminal, the second call establishment request message and the first call establishment request message satisfying at least one of the following: a speech coding mode carried by the second call establishment request message being different from a speech coding mode carried by the first call establishment request message, the number of parameters included in a first message header carried by the second call establishment request message being different from the number of parameters included in a first message header carried by the first call establishment request message and, when the first call establishment request message carries a second message header, the second call establishment request message not carrying a second message header.

This application relates to a speech coding method, an electronic device, and a storage medium. The method includes: extracting a first speech frame feature corresponding to a first to-be-encoded speech frame, and obtaining a first speech frame criticality level corresponding to the first to-be-encoded speech frame based on the first speech frame feature; extracting a second speech frame feature corresponding to the subsequent speech frame, and obtaining a second speech frame criticality level corresponding to the subsequent speech frame based on the second speech frame feature; obtaining a criticality trend feature based on the first speech frame criticality level and the second speech frame criticality level, and determining, using the criticality trend feature, an encoding bit rate corresponding to the to-be-encoded speech frame; and encoding the first to-be-encoded speech frame based on the encoding bit rate to obtain an encoding result.

The invention discloses a real-time voice recognition method, related equipment and a computer program product, and the method carries out the audio coding of real-time voice stream data to be recognized, and obtains an audio coding feature as the basis of the subsequent voice recognition. The real-time voice coding module carries out coding processing on the audio coding features, and an acoustic vector dt of the token to be decoded at the current decoding moment can be obtained. The real-time speech coding module adopted by the invention is a coding module in a real-time speech recognition model for token-by-token prediction, so that the acoustic boundary of the current token to be decoded can be determined, and the acoustic vector dt is obtained. According to the method, the acoustic vector dt is further mapped to a large model input space, it is guaranteed that the vector dimension meets the input dimension requirement of the large model, the mapped acoustic vector dt is sent to the large model for forward reasoning calculation, and the real-time speech recognition effect can be improved by means of the powerful context modeling capacity and rich text field knowledge of the large model.

The invention belongs to the field of natural language processing, and relates to a hatred recognition method combining multi-layer hatred generation and progressive comparative learning, which comprises the following steps: acquiring a private hatred recognition model trained by speech input to obtain speech characteristics, and obtaining a hatred recognition result according to the speech characteristics; the training process of the hatred recognition model comprises the following steps: acquiring a speech set, and inputting the speech set into a feature coding module to obtain a speech coding feature fi; the speech coding characteristics comprise hatred speech coding characteristics fih and normal speech coding characteristics fin; inputting the characteristic fih into a multi-layer hatred generation module to obtain a multi-layer hatred speech Xih; inputting the speech Xih into a feature extraction module to obtain a multi-layer hatred speech feature Fih; calculating a total loss function value according to the features Fih and fi, and updating model parameters according to the total loss function value until a trained model is obtained; according to the method, different layers of hatred expression modes are fully covered through the multi-layer hatred generation module, so that various expressions can be accurately recognized, and the recognition robustness is improved.

The embodiment of the invention provides a voice synthesis method and device, equipment, a storage medium and a program product. The method includes obtaining a speech coding representation corresponding to an initial speech of a first speaker, the speech coding representation indicating speech features unrelated to the speaker, the initial speech including speech content information. A pitch predictive encoded representation related to the initial speech is generated using a predictor model based on the speech encoded representation and a reference speech encoded representation, the reference speech encoded representation indicating a speech encoded representation corresponding to the second speaker, the pitch predictive encoded representation indicating pitch information of the second speaker. And generating a target speech corresponding to a second speaker by using a speech decoder model based on the pitch prediction coded representation and the reference speech coded representation, a speech signal of the second speaker including speech content information. In this way, the quality of the generated target voice is improved.

The invention discloses a robust speech recognition method and system based on multi-stage feature fusion, and relates to the technical field of speech recognition. The invention provides a speech recognition model for robust speech recognition, which comprises the following steps: firstly, extracting a magnitude spectrum | Y | from noisy speech Y through a speech coding part, coding the magnitude spectrum | Y | into a preliminary feature Ybasic, then, processing the Ybasic through a speech enhancement part to obtain a hidden feature Yhidden, a masking feature Ymask and a mapping feature Ymap, and finally, carrying out speech recognition on the hidden feature Yhidden, the masking feature Ymask and the mapping feature Ymap. Then, a fusion feature Ffuse is obtained through three-stage feature fusion, and finally, the Ffuse is decoded through a voice decoder to obtain Result. According to the model, feature complementation, feature interaction and high-low layer semantic alignment in the process are enhanced, the problem of voice distortion introduced by voice enhancement is systematically relieved, information loss in different stages is solved, and therefore the final voice recognition effect is guaranteed.

The invention discloses a lightweight semantic preserving coding and decoding and voice signal reconstruction method, which comprises the following steps of: acquiring an analog audio signal from an environment, inputting an original audio signal and an audio signal subjected to high-frequency filtering to two ends of a comparator, outputting an ultralow-bit-rate binary bit stream and transmitting the ultralow-bit-rate binary bit stream to a receiving end, and after the receiving end receives the signal, outputting the ultralow-bit-rate binary bit stream to the voice signal reconstruction end. The method comprises the following steps: inputting a voice signal to an end-to-end U-Net convolutional neural network of a bottleneck layer integrated LSTM module, outputting a reconstructed voice signal, training the network by using voice coding sample reconstruction loss and multi-scale short-time Fourier transform, and performing voice signal reconstruction on an ultra-low bit rate binary bit stream of a to-be-reconstructed audio after training. The technical contradiction that in resource-limited remote audio collection and distributed sensing tasks, node hardware resources are extremely limited, and ultra-low bit rate signal compression distortion causes that node microminiaturization deployment and high voice reconstruction quality cannot be considered at the same time is solved.

This application relates to an automatic speech code recognition method, device, and medium based on natural language processing, belonging to the field of voice communication technology. The automatic speech code recognition method based on natural language processing includes the following steps: S1, selecting a speech code; S2, restoring the speech information to a standard speech code speech data stream according to the selected speech code format; S3, obtaining the language matching information of the speech data stream; if the speech data stream does not meet the requirements for all languages ​​under the current speech code format, repeating steps S1-S3; if the speech data stream meets the requirements for any language under the current speech code format, proceeding to the next step; S4, obtaining the current speech code format, and outputting the speech information as a speech data stream in the current speech code format. This application can quickly identify speech codes and ensure smooth communication during communication failures.

The present application discloses a speech filtering method, device, storage medium and equipment, which belongs to the field of speech coding and decoding technology. The method mainly includes: encoding the speech signal according to a standard Bluetooth encoder without a post-filtering module, and decoding the encoded speech signal to a transform domain noise shaping decoding module according to a standard decoder without a post-filtering module to obtain speech spectrum coefficients; inputting the speech spectrum coefficients into a pre-trained neural network model to obtain target spectrum coefficients corresponding to the speech spectrum coefficients; and according to the remaining decoding steps of the standard decoder without a post-filtering module, inputting the target spectrum coefficients into the low-latency improved inverse discrete cosine transform module of the standard decoder to obtain the target speech signal corresponding to the target spectrum coefficients. The present application omits the complex post-filtering operation in the Bluetooth encoding process, and only uses the pre-trained neural network model for filtering in the Bluetooth decoding process, so that it achieves a sound quality close to that of standard decoding.

The invention relates to a voice code automatic identification method and device based on natural language processing and a medium, and relates to the technical field of voice communication, and the voice code automatic identification method based on natural language processing comprises the following steps: S1, selecting a voice code; s2, restoring the voice information into a voice data stream of standard voice coding according to the selected voice coding format; s3, obtaining a language condition that the voice data stream meets a requirement; if the voice data streams in all languages do not meet the requirements in the current voice coding format, repeating the steps S1 to S3; if the voice data stream in any language meets the requirement in the current voice coding format, executing the next step; and S4, acquiring a current voice coding format, and outputting a voice data stream by the voice information in the current voice coding format. The voice codes can be quickly recognized when communication is abnormal, and smooth communication is guaranteed.

1. The name of the design product: AI edge server (B033). 2. The use of the design product: used to carry intelligent visual deep learning processor, through the combination of various open source large models, realize general large model, text to image, text to search image, text to search video, speech to text, text to speech, code generation and other applications. 3. The design points of the design product: in shape. 4. The picture or photo that best indicates the design points: perspective view 1.

The invention relates to the technical field of speech semantics, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a speech emotion recognition method, device, equipment and medium based on a target scene, and the method comprises the steps: obtaining an initial speech signal, carrying out the signal coding of the initial speech signal, and obtaining a speech coding feature; performing acoustic feature extraction on the voice coding features to obtain initial acoustic features; performing feature hierarchy division on the initial acoustic feature to obtain a plurality of voice hierarchy features, and performing hierarchical attention processing on the plurality of voice hierarchy features to obtain a target voice feature; performing emotion feature extraction on the target voice feature to obtain a plurality of voice emotion features; and obtaining a target voice scene, mapping the plurality of voice emotion features to the target voice scene, and identifying a target voice emotion track in the mapped target voice scene. According to the invention, the accuracy and efficiency of speech emotion recognition are improved.

The present invention discloses a scene-based speech recognition method and device. The method includes: obtaining speech coding features of a speech signal to be recognized; obtaining scene keywords corresponding to the application scene of the speech signal to be recognized, and generating a scene feature vector containing context dependency based on the scene keywords; using a multi-layer bidirectional neural network, using context features of different depths layer by layer to extract deep hidden features related to the scene keywords from the scene feature vector; based on a dynamic attention mechanism, fusing the deep hidden features with the speech coding features to generate fused features containing the scene keywords; based on the fused features, calculating the probability distribution of the speech units of the speech signal to be recognized through a classifier, and classifying the speech units based on the probability distribution; performing speech recognition on the speech signal to be recognized according to the classification results of the speech units. The present invention solves the technical problem of inaccurate speech recognition in the prior art.

The application provides a neural network-based lightweight streaming speech coding system and method, which comprises an encoding compression end and a decoding reconstruction end; the encoding compression end comprises a speech encoder and a quantizer; the speech encoder comprises a time-frequency conversion module, a feature extraction module, an encoding end channel conversion module and a long-range time domain correlation extraction module; the feature extraction module comprises a time-frequency feature extraction submodule and a time-frequency size downsampling submodule; the quantizer comprises two or more layers of quantization modules, each layer of quantization module comprising an encoding end first domain conversion module, an encoding end vector quantization module, an encoding end inverse vector quantization module and an encoding end second domain conversion module; and the decoding reconstruction end comprises an inverse quantizer and a speech decoder. The application can effectively extract multi-scale feature information in a speech signal, so that high-quality speech compression coding and reconstruction can be realized at an extremely low code rate, and the parameter quantity is small, the complexity is low, and streaming coding and decoding can be realized.

An apparatus includes a memory configured to store data associated with a machine learning (ML) based speech synthesis model. The apparatus also includes a speech encoder including the ML based speech synthesis model. The speech encoder is configured to perform a synthetic analysis operation of an input speech signal including generating a synthetic version of the input speech signal by the ML based speech synthesis model.

A method (600) includes receiving training data (301) including a plurality of sets of text-to-speech (TTS) spoken utterances (510), each set of TTS spoken utterances being associated with a respective language and including TTS utterances of synthetic speech including a corresponding reference speech representation (504) paired with a corresponding input text sequence (502). For each TTS utterance, the method includes generating a corresponding TTS encoded text representation for the corresponding input text sequence (512); generating a corresponding speech code for the TTS utterance of the corresponding synthetic speech (514); generating a shared encoder output (532, 534); generating a predicted speech representation for the TTS utterance of the corresponding synthetic speech (522); and determining a reconstruction loss (545). The method further includes training a TTS model based on the reconstruction loss determined for the TTS utterance in each set of TTS spoken training utterances (501).

The application discloses an adaptive speech coding adjustment method and device, equipment and medium, and the method is applied to a talking stage of a voice communication system. The application detects the voice quality under the current channel condition by using a real-time intelligent voice measurement algorithm, and adaptively adjusts the speech coding type based on the voice quality detection result, so that the voice quality of the conversation is improved, and the user experience is improved.