30 results about "Phoneme recognition" patented technology

The invention belongs to the field of speech recognition and speech evaluation, and provides an English auxiliary pronunciation training method and system based on multi-task learning, and the method comprises the steps: obtaining a to-be-evaluated English pronunciation audio; preprocessing the English pronunciation audio, extracting audio features, and obtaining depth feature vectors of English pronunciation; performing multi-granularity evaluation on the depth feature vector by using a pre-trained automatic pronunciation evaluation model, and determining phoneme-level, word-level and sentence-level pronunciation evaluation results; performing phoneme recognition on the depth feature vector by using a pre-trained pronunciation detection and diagnosis model, and calculating a phoneme error rate of English pronunciation; in the training process of the automatic pronunciation evaluation model and the pronunciation detection and diagnosis model, the loss of automatic pronunciation evaluation and wrong pronunciation detection and diagnosis is jointly optimized. According to the method, the accuracy of pronunciation evaluation is improved by fully utilizing the correlation between automatic pronunciation evaluation and wrong pronunciation detection and diagnosis tasks.

The invention provides a digital human lip shape synchronization method based on voice driving. The method comprises the following steps: acquiring a voice signal; voice features related to lip shape changes in the voice signals are extracted; mapping the extracted voice features into lip shape change data; and applying the lip shape change data to the digital human model and carrying out real-time rendering to finish lip shape synchronization. According to the method, the voice signal characteristics are directly analyzed, subtle changes of pronunciation can be more accurately captured, and more natural and smooth lip-shaped actions are generated; the method is suitable for a multi-language environment, and especially improves the processing effect of tone languages such as Chinese and the like. An optimized signal processing and mapping algorithm improves the lip shape synchronization speed, and the real-time interaction requirement is met; a complicated phoneme identification process is not needed, and the technical implementation difficulty and the maintenance cost are reduced; through more accurate and natural lip shape synchronization, the overall performance and interaction naturalness of the digital human are remarkably improved.

The invention relates to the field of artificial intelligence and the field of financial science and technology, and discloses a response strategy generation and execution method, which comprises the following steps of: constructing a multi-task learning structure based on an enhanced spatio-temporal feature extraction model by collecting voice signals of a user and extracting emotional features; sentiment classification, sentiment intensity regression, phoneme recognition and gender recognition are used as joint tasks for training, and a multi-task learning model is obtained. After emotion features are input into the model, emotion category labels and emotion intensity are generated, a strategy table is inquired in combination with a current application scene, and a corresponding response strategy is determined and executed. Through the construction of the multi-task learning model and the joint task training, the emotion category and emotion intensity of the user are accurately identified, and the response strategy is dynamically adjusted in combination with the application scene, so that the emotion perception ability and the personalized service effect of the smart home system are improved, and the user experience and the interaction effect are remarkably improved.

The invention provides a spoken language pronunciation scoring method and system and a spoken language pronunciation scoring all-in-one machine, and relates to the technical field of spoken language evaluation, and the method comprises the steps: firstly obtaining a scoring task text, a to-be-scored voice, and an evaluation text corresponding to the to-be-scored voice; and then extracting voice features of the to-be-scored voice, and inputting the scoring task text, the voice features and the evaluation text into the evaluation model to obtain an evaluation score for the to-be-scored voice output by the evaluation model. According to the method, multi-modal input is provided for the evaluation model, the to-be-scored voice can be evaluated without participation of a phoneme recognition module, the evaluation score is further obtained, the scoring process is simple, the calculation overhead can be reduced, and the pronunciation scoring efficiency can be improved. And moreover, by utilizing a single model, the accuracy of the evaluation score can be improved. The method can be widely applied to spoken language evaluation services such as learning machines and teaching and examination platforms.

The invention discloses a spoken language evaluation system and method based on multi-dimensional factors, and relates to the technical field of data processing, and the method comprises the steps: obtaining an audio text, a phoneme list and a phoneme time boundary list corresponding to voice data according to a pre-trained text phoneme recognition model; calculating a fluency score according to the phoneme list and the phoneme time boundary list; obtaining a semantic score of the audio text through a pre-trained semantic model; obtaining a grammar score of the audio text through a pre-trained grammar model; matching the audio text and the answer text according to a text matching method based on an editing distance to obtain a text matching score; the pronunciation score of the audio data is finally obtained through the phoneme confidence of the phoneme list; dynamically adjusting the weight of the score of each dimension according to the length of the audio text, and calculating a final spoken language evaluation score; through multi-dimensional evaluation, the spoken language ability is comprehensively evaluated according to dynamic weight adjustment of the scene, and comprehensiveness, fairness, accuracy and high efficiency of spoken language evaluation are achieved.

A method is provided for identifying synthetic “deepfake” audio samples versus organic audio samples. Methods may include: receiving an audio sample comprising speech; converting the speech to text; aligning the text with phonemes identified within the audio sample; filtering the audio sample to only contain predetermined phonemes; obtaining, from the audio sample, a frequency response vector for each of the predetermined phonemes; transforming the frequency response vector for each of the predetermined phonemes to a classification space vector for each of the predetermined phonemes having a magnitude; normalizing the classification space vector for each of the predetermined phonemes; identifying each of the predetermined phonemes as one of synthetic or organic based on the classification space vector for each of the predetermined phonemes; and identifying the audio sample as synthetic or organic based on identification of each of the predetermined phonemes as one of synthetic or organic.

The invention discloses a speech recognition method and device based on phoneme error correction, a medium and equipment, and the method comprises the steps: carrying out the sound feature extraction of a to-be-recognized speech signal, and obtaining an audio signal feature; performing phoneme recognition based on the audio signal features to obtain a phoneme sequence; performing speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence; and performing text mapping processing on the phoneme recognition sequence to obtain a speech recognition result. According to the method disclosed by the invention, the influence of wrong pronunciation on speech recognition is solved by correcting the phonemes, so that the recognition result is closer to real text information, and the speech recognition accuracy of the speaker is effectively improved.

The invention relates to an English phoneme m recognition method based on a spline interpolation wavelet neural network, and solves the problems that a traditional recognition method is low in calculation speed and low in precision and stability. Audio input is converted into an analyzed discrete signal set by using a sample-and-hold technology, and a standardized audio data set is obtained through normalization and threshold processing to serve as input of a neural network; a six-order spline interpolation wavelet is selected, and a criterion function of a neural network is constructed according to mapping coefficients of the interpolation wavelet and a common wavelet; taking a wavelet as an excitation function of the neural network; constructing a training feedback matrix by using the mapping coefficients of the interpolation wavelet and the common wavelet; the wavelet neural network training has global convergence, and an output layer weight is a corresponding wavelet coefficient; performing discrete Fourier transform on the wavelet coefficient to obtain a frequency spectrum of the wavelet coefficient, and identifying a feature point and a distribution characteristic thereof as a signal identification condition; and judging whether the input phonemes contain consonants m or not by comparing the feature points with the distribution characteristics of the feature points.

The present invention relates to a language recognition method based on phoneme-based language discriminative features, belonging to the technical field of audio signal processing. First, the present invention extracts a phoneme set from the TIMIT dataset, constructs phoneme phonetic discriminative features for the phoneme set, trains and tests a phoneme recognizer using the phoneme phonetic discriminative features, and outputs the frame-level phoneme probability vector of the audio; then, it obtains multi-language corpora from the LibriVox audio database, expands the phoneme set extracted from the TIMIT dataset for the multi-language corpora, and outputs the frame phoneme probability features of the short-time complete semantic speech segments of the languages; finally, it constructs the phoneme probability features of the speech segments according to the frame phoneme probability features of different languages output by the phoneme recognizer, and further constructs the language discriminative features of the speech segments. The present invention can perform language recognition in a classical two-dimensional convolutional neural network and obtain language recognition results with a high recognition rate.

The present invention belongs to the field of speech recognition and speech evaluation, and provides an English auxiliary pronunciation training method and system based on multi-task learning, which obtains English pronunciation audio to be evaluated; pre-processes the English pronunciation audio, extracts audio features, and obtains a deep feature vector of English pronunciation; uses a pre-trained automatic pronunciation evaluation model to perform multi-granularity evaluation on the deep feature vector to determine pronunciation evaluation results at the phoneme level, word level, and sentence level; uses a pre-trained pronunciation detection and diagnosis model to perform phoneme recognition on the deep feature vector and calculate the phoneme error rate of English pronunciation; in the training process of the automatic pronunciation evaluation model and the pronunciation detection and diagnosis model, jointly optimizes the loss of automatic pronunciation evaluation and wrong pronunciation detection and diagnosis. The present invention makes full use of the correlation between the automatic pronunciation evaluation and wrong pronunciation detection and diagnosis tasks, and improves the accuracy of pronunciation evaluation.

The invention relates to the technical field of speech recognition, and discloses a phoneme recognition model training method, a phoneme recognition method and a phoneme recognition device.A first sample phoneme sequence corresponding to normal speech data is recognized through a first phoneme recognition model, and a second sample phoneme sequence corresponding to dysarthria speech data is recognized through a second phoneme recognition model; and taking the second sample phoneme sequence as a reference phoneme sequence in a scene without a fixed text. According to the method, fine tuning training is performed on the basis of the first phoneme recognition model according to the individual dysarthria data of the dysarthria patient to obtain the individualized second phoneme recognition model, and the situation of free dialogue or poor text compliance can be covered even in a scene without a fixed text, so that the accuracy of recognition of the dysarthria is improved. The actual pronunciation ability of the patient can be reflected in a natural communication context, dependence on fixed texts and artificial phoneme level alignment labeling is avoided, higher flexibility and adaptability are achieved, and a reliable basis can be provided for rehabilitation training and curative effect tracking.

The invention relates to a phoneme-based speech recognition method and device of a pronunciation-free dictionary, computer equipment and a storage medium. The method is applied to a server, a speech recognition framework is deployed in the server, the speech recognition framework comprises a phoneme recognition model, a phoneme-to-character conversion model and an auxiliary character-to-phoneme conversion model, and the method comprises the following steps: decoding speech information to be recognized based on the phoneme recognition model to generate a plurality of candidate phoneme sequences; decoding the plurality of candidate phoneme sequences based on a phoneme-word conversion model to generate a plurality of candidate text sequences; scoring the plurality of candidate text sequences according to an auxiliary character-to-pronunciation conversion model to obtain a speech recognition result; wherein the speech recognition framework is obtained by training an initial speech recognition framework based on a joint random approximation (JSA) algorithm in advance. And the accuracy of speech recognition is improved.

Disclosed are a speech recognition method, apparatus, device and storage medium. The method comprises: obtaining acoustic features of a speech signal, wherein the speech signal comprises a plurality of speech frames; inputting the acoustic features into a convolutional network to obtain convolutional features of the acoustic features; inputting the convolutional features into a long short-term memory (LSTM) recurrent neural network to obtain associated features of a plurality of speech frames output by the LSTM recurrent neural network; and obtaining a phoneme recognition result of a first speech frame before the set time length according to the associated features of a plurality of speech frames within a set time length.

The application discloses a virtual anchor real-time driving system based on facial motion capture, aims to solve the problems of insufficient precision, lack of naturalness and weak scene adaptation of virtual anchor facial motion in the prior art, and through multi-thread parallel collection of audio and video streams, optimizes data quality through adaptive preprocessing; adopts multi-dimensional facial feature collaborative extraction, double-branch phoneme recognition, face function area semantic segmentation and confidence dynamic weight adjustment technology, realizes accurate representation and collaborative fusion of expression and lip feature; finally, through the lip-expression collaborative driving mechanism, the real-time driving characteristics of the adaptive virtual image are output; thereby effectively improving the precision and naturalness of virtual anchor facial motion, enhancing the complex scene adaptability, giving consideration to real-time response and low-cost deployment, and being applicable to virtual live broadcast, online education and other scenes, and having good application value.

A method for phoneme identification. The method includes receiving an audio signal from a speaker, performing initial processing comprising filtering the audio signal to remove audio features, the initial processing resulting in a modified audio signal, transmitting the modified audio signal to a phoneme identification method and a phoneme replacement method to further process the modified audio signal, and transmitting the modified audio signal to a speaker. Also, a system for identifying and processing audio signals. The system includes at least one speaker, at least one microphone, and at least one processor, wherein the processor processes audio signals received using a method for phoneme replacement.

The invention relates to a method, a device, equipment and a medium for automatically labeling phoneme labels for voice, and the method comprises the steps: obtaining a decoded image, recognizing a to-be-labeled audio based on a preset phoneme recognition model, and generating a recognition result, the preset phoneme recognition model is used for calculating the acoustic score of each audio frame in the to-be-labeled audio, an identification result is generated according to the acoustic score; the acoustic score is used for indicating a probability value that each audio frame is a preset phoneme frame, and the identification result is used for indicating a distribution condition of each audio frame in the to-be-labeled audio, and generating a phoneme label of the to-be-labeled audio according to the identification result and the decoded image, the phoneme label is used for indicating at least one of the start time, the end time and the duration of each phoneme in the audio to be labeled. Therefore, the recognition result of the preset phoneme recognition model is corrected based on the constraint condition in the decoded image, the phoneme tag of the audio is generated, and the accuracy of the phoneme tag corresponding to the audio data is improved.

A speech recognition method includes: performing phoneme recognition on a speech signal to obtain a phoneme recognition result corresponding to a speech frame in the speech signal, the phoneme recognition result indicating a probability distribution of the corresponding speech frame in a phoneme space, and the phoneme space comprising a plurality of phonemes and a blank output; performing suppression adjustment on a probability of the blank output in the phoneme recognition result corresponding to the speech frame, to reduce a ratio of the probability of the blank output to a probability of a phoneme in the phoneme recognition result; and inputting the adjusted phoneme recognition result corresponding to the speech frame into a decoding map to obtain a recognition text sequence corresponding to the speech signal, the decoding map comprising a mapping relationship between characters and phonemes.

The present application provides a method, device, electronic device and storage medium for recognizing instruction words, which belongs to the field of speech recognition technology. The present application obtains a speech to be recognized and an instruction word set, wherein the instruction word set includes at least one instruction word and an instruction word phoneme sequence converted from the text content corresponding to the instruction word; inputs the speech to be recognized into a pre-trained phoneme recognition model so that the phoneme recognition model outputs the target phoneme corresponding to the speech to be recognized; predicts the target phoneme sequence corresponding to the target phoneme; determines the target instruction word phoneme sequence corresponding to the target phoneme sequence according to the instruction word set, and uses the instruction word corresponding to the target instruction word phoneme sequence as the target instruction word corresponding to the speech to be recognized.

The application discloses a spoken language evaluation system and method based on multiple dimensions, and relates to the technical field of data processing.The system comprises the following steps: obtaining audio text, a phoneme list and a phoneme time boundary list corresponding to speech data according to a pre-trained text phoneme recognition model; calculating a fluency score according to the phoneme list and the phoneme time boundary list; obtaining a semantic score of the audio text through a pre-trained semantic model; obtaining a syntax score of the audio text through a pre-trained syntax model; matching the audio text and an answer text according to a text matching method based on an edit distance to obtain a text matching score; obtaining a pronunciation score of the audio data finally through phoneme confidence of the phoneme list; dynamically adjusting the weight of each dimension score according to the length of the audio text, and calculating a final spoken language evaluation score; and the application comprehensively evaluates the spoken language ability through multi-dimensional evaluation and dynamic weight adjustment according to the scene, and realizes the comprehensiveness, fairness, accuracy and high efficiency of the spoken language evaluation.

A speech adaptive completion system based on a multimodal knowledge graph includes: a data receiver, a data analyzer and a data reasoner, wherein: the data receiver performs preprocessing according to the received audio and video data and outputs it to the data analyzer; the data analyzer extracts waveform timing features and lip trajectory features from the analysis of speech and images, and obtains a phoneme sequence through multimodal joint representation; the data reasoner performs domain conversation modeling and candidate text prediction based on historical texts, performs text reasoning in combination with phoneme sequences, obtains sentences with semantics, and synthesizes completed speech based on waveform features. The present invention uses a phoneme reasoning model to perform phoneme recognition when the speech modality is missing, and performs domain conversation modeling on the historical text generated by the existing speech based on the semantic relationship between entities in the multimodal knowledge graph, thereby reasoning and generating text with semantics, and synthesizing the speech in combination with the waveform features of the user's speech to form the completed audio.

The invention discloses a voice backdoor attack method based on phoneme recognition. The method comprises the following steps: carrying out phoneme recognition on all to-be-trained voice samples, and extracting a corresponding phoneme table of each voice sample; self-adaptive amplitude enhancement is carried out on phonemes needing to be enhanced in the voice sample; the frequency intensity of the voice data set is calculated, a frequency initialization pure tone trigger with the lowest frequency intensity is selected, the trigger is implanted into the voice sample, and a label is modified into a target label; the voiced voice samples are mixed into the benign data set in proportion to obtain a voiced data set, and a victim model trained by using the voiced data set is provided with an implanted backdoor. According to the method, on the basis of ensuring the concealment of the trigger, the effectiveness of the backdoor attack of the speech recognition model is improved, and the method has a small influence on the accuracy of benign sample recognition.

The invention discloses a virtual anchor real-time driving system based on facial motion capture, and aims at solving the problems that in the prior art, virtual anchor facial motions are insufficient in accuracy, lack of natural sense and weak in scene adaptation. Audio and video streams are collected in parallel through multiple threads, and the data quality is optimized through self-adaptive preprocessing; the technologies of multi-dimensional facial feature collaborative extraction, double-branch phoneme recognition, face functional region semantic segmentation and confidence coefficient dynamic weight adjustment are adopted, and accurate representation and collaborative fusion of expression and mouth shape features are achieved; and finally, outputting real-time driving characteristics matched with the virtual image through a mouth shape-expression cooperative driving mechanism. Therefore, the accuracy and the natural sense of the face action of the virtual anchor are effectively improved, the adaptability of complex scenes is enhanced, real-time response and low-cost deployment are both considered, and the method is suitable for scenes such as virtual live broadcast and online education and has good application value.

The present application relates to the field of audio analysis, and discloses a kind of audio data's tone analysis and play method, equipment and storage medium.The method comprises: receiving audio data;According to preset phoneme recognition algorithm, the phoneme recognition processing is carried out to the audio data, and phoneme sequence is obtained;According to preset tone rhythm data, the phoneme sequence is divided and marked, and tone sequence is obtained;Using preset description library, the tone sequence is described and matched, and the description text corresponding to the audio data is obtained;When the audio data plays, based on the playing position of the audio data, the text field corresponding to the playing position in the description text is shown.In the embodiment of the present application, the technical problem that the current analysis technology lacks tone analysis of audio and cannot meet the demand of people for audio music appreciation is solved.

The present invention provides a phoneme recognition method, apparatus, electronic device, and storage medium. The method includes: determining the speech to be recognized; inputting the speech to be recognized into a phoneme recognition model to obtain a phoneme recognition result output by the phoneme recognition model; the phoneme recognition model is obtained by training a first recognition model based on sample speeches of multiple languages and phoneme-level labels of each sample speech, and the first recognition model is obtained by screening phoneme nodes in a second recognition model based on the similarity between phonemes corresponding to each phoneme node in the second recognition model, and the second recognition model includes phoneme nodes corresponding to multiple languages respectively. The phoneme recognition method, apparatus, electronic device, and storage medium provided by the present invention not only reduce the scale of the phoneme recognition model, but also the phoneme recognition model can accurately distinguish phonemes of different languages.

Embodiments of the present disclosure provide a speech phoneme recognition method, medium, device and computing device. The method comprises: inputting speech data to be recognized into a pre-trained phoneme recognition model, outputting a phoneme sequence corresponding to the speech data, and the phoneme sequence comprising the occurrence order of each phoneme in the speech data. The present disclosure solves the problem that speech recognition in the related art cannot effectively match the lip movement of a virtual character image and speech. The speech data is disassembled into a set of phonemes that can be represented by the lip movement of AI, and is represented in a sequence form, so that the AI can perform representation through lip movement in turn according to the duration of each phoneme by reading the phoneme sequence, thereby achieving accurate matching of lip movement and speech, and significantly improving the experience of the audience.

This application provides a digital human mouth shape driving method, system, and medium based on phoneme recognition. The method includes: acquiring input raw speech signals and processing the raw speech signals; performing feature extraction and phoneme recognition on preprocessed data information to obtain accurate phoneme category and time sequence information; constructing a preset phoneme and mouth shape parameter mapping library, the mapping library including mouth shape parameters and muscle deformation parameters corresponding to different phonemes and different pronunciation styles; dynamically matching the phoneme time sequence information with the mapping library to obtain corresponding mouth shape driving parameters and muscle deformation parameters; inputting a digital human mouth and surrounding muscle model, driving the model to deform, obtaining the driving result, and performing synchronous processing of mouth shape and speech pronunciation on the model to obtain synchronization result information; through phoneme-level fine-grained recognition, dynamic mapping matching, and precise driving of muscle models, real-time synchronization of mouth shape and speech pronunciation is achieved, improving the naturalness and realism of digital human voice interaction.

The invention relates to a learning evaluation method for English word pronunciation through audio recognition, which comprises the following steps: constructing a speech recognition system, preprocessing received user pronunciation audio, and then realizing phonetic symbol recognition of recognized words through vowel phonetic symbol recognition and screening and consonant phonetic symbol recognition and screening or full phoneme recognition. The word recognized by the final recognition phonetic symbol sequence obtained by the speech recognition system is compared with the stored standard pronunciation of the word, the pronunciation of the practicer is scored, and specific practice suggestions are given.

Techniques for enabling voice-based language learning are described. A system of the present disclosure supports receipt of speech inputs in a user's native language and a language the user is learning. The system runs an automatic speech recognition (ASR) component configured for transcribing speech in the native language (e.g., English) and runs a phoneme recognition component configured for recognizing phonemes of the learning language (e.g., Spanish). The system can respond to commands provided in the native language (e.g., “Stop lesson”, “Repeat lesson”, etc.) and provide pronunciation feedback with respect to spoken inputs in the learning language.

The invention relates to the technical field of data processing, and discloses a data calibration method and device, equipment and a medium, and the method comprises the steps: carrying out the phoneme recognition of a target audio through a target phoneme recognition model, obtaining the prediction phoneme and prediction probability of each audio frame, and obtaining the prediction probability of each audio frame according to a preset mapping table; determining a mapping phoneme of each sub-word in the transcription text corresponding to the target audio, for any audio frame, according to the predicted phoneme of the audio frame, the prediction probability corresponding to the predicted phoneme and the mapping phoneme of each sub-word, respectively matching the audio frame with each sub-word, and according to a matching result, determining a target sub-word corresponding to the audio frame, and according to the mapping phonemes of the target sub-words, the predicted phonemes of the audio frames are calibrated, and calibration results of the predicted phonemes of all the audio frames are fused to form a final calibration result. The method can be applied to the field of financial science and technology, the accuracy of phoneme recognition is improved, and a more reliable acoustic representation basis is provided for a downstream speech processing task of phoneme recognition.

The present disclosure relates to an audio analysis method, device, equipment and storage medium, and relates to the field of computer technology, and is used to solve the problem that general technologies have low accuracy and high cost when processing audio data of a terminal. The audio analysis method includes: obtaining dry audio data of a target audio; inputting the dry audio data into a target phoneme recognition model for phoneme recognition processing to obtain a target phoneme time series; the target phoneme recognition model is trained based on a phoneme set after homophonic merging of multiple languages; inputting the dry audio data into a target text recognition model for text recognition processing to obtain a target text time series; the target text recognition model is trained based on the lyrics information of the target audio; determining audio signal feature information of the dry audio data; and determining the audio analysis result of the dry audio data based on the target phoneme time series, the target text time series and the audio signal feature information.