20 results about "Phoneme recognition" patented technology

The invention provides a spoken language pronunciation scoring method and system and a spoken language pronunciation scoring all-in-one machine, and relates to the technical field of spoken language evaluation, and the method comprises the steps: firstly obtaining a scoring task text, a to-be-scored voice, and an evaluation text corresponding to the to-be-scored voice; and then extracting voice features of the to-be-scored voice, and inputting the scoring task text, the voice features and the evaluation text into the evaluation model to obtain an evaluation score for the to-be-scored voice output by the evaluation model. According to the method, multi-modal input is provided for the evaluation model, the to-be-scored voice can be evaluated without participation of a phoneme recognition module, the evaluation score is further obtained, the scoring process is simple, the calculation overhead can be reduced, and the pronunciation scoring efficiency can be improved. And moreover, by utilizing a single model, the accuracy of the evaluation score can be improved. The method can be widely applied to spoken language evaluation services such as learning machines and teaching and examination platforms.

The invention discloses a spoken language evaluation system and method based on multi-dimensional factors, and relates to the technical field of data processing, and the method comprises the steps: obtaining an audio text, a phoneme list and a phoneme time boundary list corresponding to voice data according to a pre-trained text phoneme recognition model; calculating a fluency score according to the phoneme list and the phoneme time boundary list; obtaining a semantic score of the audio text through a pre-trained semantic model; obtaining a grammar score of the audio text through a pre-trained grammar model; matching the audio text and the answer text according to a text matching method based on an editing distance to obtain a text matching score; the pronunciation score of the audio data is finally obtained through the phoneme confidence of the phoneme list; dynamically adjusting the weight of the score of each dimension according to the length of the audio text, and calculating a final spoken language evaluation score; through multi-dimensional evaluation, the spoken language ability is comprehensively evaluated according to dynamic weight adjustment of the scene, and comprehensiveness, fairness, accuracy and high efficiency of spoken language evaluation are achieved.

The invention relates to an English phoneme m recognition method based on a spline interpolation wavelet neural network, and solves the problems that a traditional recognition method is low in calculation speed and low in precision and stability. Audio input is converted into an analyzed discrete signal set by using a sample-and-hold technology, and a standardized audio data set is obtained through normalization and threshold processing to serve as input of a neural network; a six-order spline interpolation wavelet is selected, and a criterion function of a neural network is constructed according to mapping coefficients of the interpolation wavelet and a common wavelet; taking a wavelet as an excitation function of the neural network; constructing a training feedback matrix by using the mapping coefficients of the interpolation wavelet and the common wavelet; the wavelet neural network training has global convergence, and an output layer weight is a corresponding wavelet coefficient; performing discrete Fourier transform on the wavelet coefficient to obtain a frequency spectrum of the wavelet coefficient, and identifying a feature point and a distribution characteristic thereof as a signal identification condition; and judging whether the input phonemes contain consonants m or not by comparing the feature points with the distribution characteristics of the feature points.

The present invention relates to a language recognition method based on phoneme-based language discriminative features, belonging to the technical field of audio signal processing. First, the present invention extracts a phoneme set from the TIMIT dataset, constructs phoneme phonetic discriminative features for the phoneme set, trains and tests a phoneme recognizer using the phoneme phonetic discriminative features, and outputs the frame-level phoneme probability vector of the audio; then, it obtains multi-language corpora from the LibriVox audio database, expands the phoneme set extracted from the TIMIT dataset for the multi-language corpora, and outputs the frame phoneme probability features of the short-time complete semantic speech segments of the languages; finally, it constructs the phoneme probability features of the speech segments according to the frame phoneme probability features of different languages output by the phoneme recognizer, and further constructs the language discriminative features of the speech segments. The present invention can perform language recognition in a classical two-dimensional convolutional neural network and obtain language recognition results with a high recognition rate.

The invention relates to the technical field of speech recognition, and discloses a phoneme recognition model training method, a phoneme recognition method and a phoneme recognition device.A first sample phoneme sequence corresponding to normal speech data is recognized through a first phoneme recognition model, and a second sample phoneme sequence corresponding to dysarthria speech data is recognized through a second phoneme recognition model; and taking the second sample phoneme sequence as a reference phoneme sequence in a scene without a fixed text. According to the method, fine tuning training is performed on the basis of the first phoneme recognition model according to the individual dysarthria data of the dysarthria patient to obtain the individualized second phoneme recognition model, and the situation of free dialogue or poor text compliance can be covered even in a scene without a fixed text, so that the accuracy of recognition of the dysarthria is improved. The actual pronunciation ability of the patient can be reflected in a natural communication context, dependence on fixed texts and artificial phoneme level alignment labeling is avoided, higher flexibility and adaptability are achieved, and a reliable basis can be provided for rehabilitation training and curative effect tracking.

The invention relates to a phoneme-based speech recognition method and device of a pronunciation-free dictionary, computer equipment and a storage medium. The method is applied to a server, a speech recognition framework is deployed in the server, the speech recognition framework comprises a phoneme recognition model, a phoneme-to-character conversion model and an auxiliary character-to-phoneme conversion model, and the method comprises the following steps: decoding speech information to be recognized based on the phoneme recognition model to generate a plurality of candidate phoneme sequences; decoding the plurality of candidate phoneme sequences based on a phoneme-word conversion model to generate a plurality of candidate text sequences; scoring the plurality of candidate text sequences according to an auxiliary character-to-pronunciation conversion model to obtain a speech recognition result; wherein the speech recognition framework is obtained by training an initial speech recognition framework based on a joint random approximation (JSA) algorithm in advance. And the accuracy of speech recognition is improved.

The application discloses a virtual anchor real-time driving system based on facial motion capture, aims to solve the problems of insufficient precision, lack of naturalness and weak scene adaptation of virtual anchor facial motion in the prior art, and through multi-thread parallel collection of audio and video streams, optimizes data quality through adaptive preprocessing; adopts multi-dimensional facial feature collaborative extraction, double-branch phoneme recognition, face function area semantic segmentation and confidence dynamic weight adjustment technology, realizes accurate representation and collaborative fusion of expression and lip feature; finally, through the lip-expression collaborative driving mechanism, the real-time driving characteristics of the adaptive virtual image are output; thereby effectively improving the precision and naturalness of virtual anchor facial motion, enhancing the complex scene adaptability, giving consideration to real-time response and low-cost deployment, and being applicable to virtual live broadcast, online education and other scenes, and having good application value.

A speech recognition method includes: performing phoneme recognition on a speech signal to obtain a phoneme recognition result corresponding to a speech frame in the speech signal, the phoneme recognition result indicating a probability distribution of the corresponding speech frame in a phoneme space, and the phoneme space comprising a plurality of phonemes and a blank output; performing suppression adjustment on a probability of the blank output in the phoneme recognition result corresponding to the speech frame, to reduce a ratio of the probability of the blank output to a probability of a phoneme in the phoneme recognition result; and inputting the adjusted phoneme recognition result corresponding to the speech frame into a decoding map to obtain a recognition text sequence corresponding to the speech signal, the decoding map comprising a mapping relationship between characters and phonemes.

The application discloses a spoken language evaluation system and method based on multiple dimensions, and relates to the technical field of data processing.The system comprises the following steps: obtaining audio text, a phoneme list and a phoneme time boundary list corresponding to speech data according to a pre-trained text phoneme recognition model; calculating a fluency score according to the phoneme list and the phoneme time boundary list; obtaining a semantic score of the audio text through a pre-trained semantic model; obtaining a syntax score of the audio text through a pre-trained syntax model; matching the audio text and an answer text according to a text matching method based on an edit distance to obtain a text matching score; obtaining a pronunciation score of the audio data finally through phoneme confidence of the phoneme list; dynamically adjusting the weight of each dimension score according to the length of the audio text, and calculating a final spoken language evaluation score; and the application comprehensively evaluates the spoken language ability through multi-dimensional evaluation and dynamic weight adjustment according to the scene, and realizes the comprehensiveness, fairness, accuracy and high efficiency of the spoken language evaluation.

A speech adaptive completion system based on a multimodal knowledge graph includes: a data receiver, a data analyzer and a data reasoner, wherein: the data receiver performs preprocessing according to the received audio and video data and outputs it to the data analyzer; the data analyzer extracts waveform timing features and lip trajectory features from the analysis of speech and images, and obtains a phoneme sequence through multimodal joint representation; the data reasoner performs domain conversation modeling and candidate text prediction based on historical texts, performs text reasoning in combination with phoneme sequences, obtains sentences with semantics, and synthesizes completed speech based on waveform features. The present invention uses a phoneme reasoning model to perform phoneme recognition when the speech modality is missing, and performs domain conversation modeling on the historical text generated by the existing speech based on the semantic relationship between entities in the multimodal knowledge graph, thereby reasoning and generating text with semantics, and synthesizing the speech in combination with the waveform features of the user's speech to form the completed audio.

The invention discloses a voice backdoor attack method based on phoneme recognition. The method comprises the following steps: carrying out phoneme recognition on all to-be-trained voice samples, and extracting a corresponding phoneme table of each voice sample; self-adaptive amplitude enhancement is carried out on phonemes needing to be enhanced in the voice sample; the frequency intensity of the voice data set is calculated, a frequency initialization pure tone trigger with the lowest frequency intensity is selected, the trigger is implanted into the voice sample, and a label is modified into a target label; the voiced voice samples are mixed into the benign data set in proportion to obtain a voiced data set, and a victim model trained by using the voiced data set is provided with an implanted backdoor. According to the method, on the basis of ensuring the concealment of the trigger, the effectiveness of the backdoor attack of the speech recognition model is improved, and the method has a small influence on the accuracy of benign sample recognition.

The invention discloses a virtual anchor real-time driving system based on facial motion capture, and aims at solving the problems that in the prior art, virtual anchor facial motions are insufficient in accuracy, lack of natural sense and weak in scene adaptation. Audio and video streams are collected in parallel through multiple threads, and the data quality is optimized through self-adaptive preprocessing; the technologies of multi-dimensional facial feature collaborative extraction, double-branch phoneme recognition, face functional region semantic segmentation and confidence coefficient dynamic weight adjustment are adopted, and accurate representation and collaborative fusion of expression and mouth shape features are achieved; and finally, outputting real-time driving characteristics matched with the virtual image through a mouth shape-expression cooperative driving mechanism. Therefore, the accuracy and the natural sense of the face action of the virtual anchor are effectively improved, the adaptability of complex scenes is enhanced, real-time response and low-cost deployment are both considered, and the method is suitable for scenes such as virtual live broadcast and online education and has good application value.

The present application relates to the field of audio analysis, and discloses a kind of audio data's tone analysis and play method, equipment and storage medium.The method comprises: receiving audio data;According to preset phoneme recognition algorithm, the phoneme recognition processing is carried out to the audio data, and phoneme sequence is obtained;According to preset tone rhythm data, the phoneme sequence is divided and marked, and tone sequence is obtained;Using preset description library, the tone sequence is described and matched, and the description text corresponding to the audio data is obtained;When the audio data plays, based on the playing position of the audio data, the text field corresponding to the playing position in the description text is shown.In the embodiment of the present application, the technical problem that the current analysis technology lacks tone analysis of audio and cannot meet the demand of people for audio music appreciation is solved.

The present invention provides a phoneme recognition method, apparatus, electronic device, and storage medium. The method includes: determining the speech to be recognized; inputting the speech to be recognized into a phoneme recognition model to obtain a phoneme recognition result output by the phoneme recognition model; the phoneme recognition model is obtained by training a first recognition model based on sample speeches of multiple languages and phoneme-level labels of each sample speech, and the first recognition model is obtained by screening phoneme nodes in a second recognition model based on the similarity between phonemes corresponding to each phoneme node in the second recognition model, and the second recognition model includes phoneme nodes corresponding to multiple languages respectively. The phoneme recognition method, apparatus, electronic device, and storage medium provided by the present invention not only reduce the scale of the phoneme recognition model, but also the phoneme recognition model can accurately distinguish phonemes of different languages.

Embodiments of the present disclosure provide a speech phoneme recognition method, medium, device and computing device. The method comprises: inputting speech data to be recognized into a pre-trained phoneme recognition model, outputting a phoneme sequence corresponding to the speech data, and the phoneme sequence comprising the occurrence order of each phoneme in the speech data. The present disclosure solves the problem that speech recognition in the related art cannot effectively match the lip movement of a virtual character image and speech. The speech data is disassembled into a set of phonemes that can be represented by the lip movement of AI, and is represented in a sequence form, so that the AI can perform representation through lip movement in turn according to the duration of each phoneme by reading the phoneme sequence, thereby achieving accurate matching of lip movement and speech, and significantly improving the experience of the audience.

This application provides a digital human mouth shape driving method, system, and medium based on phoneme recognition. The method includes: acquiring input raw speech signals and processing the raw speech signals; performing feature extraction and phoneme recognition on preprocessed data information to obtain accurate phoneme category and time sequence information; constructing a preset phoneme and mouth shape parameter mapping library, the mapping library including mouth shape parameters and muscle deformation parameters corresponding to different phonemes and different pronunciation styles; dynamically matching the phoneme time sequence information with the mapping library to obtain corresponding mouth shape driving parameters and muscle deformation parameters; inputting a digital human mouth and surrounding muscle model, driving the model to deform, obtaining the driving result, and performing synchronous processing of mouth shape and speech pronunciation on the model to obtain synchronization result information; through phoneme-level fine-grained recognition, dynamic mapping matching, and precise driving of muscle models, real-time synchronization of mouth shape and speech pronunciation is achieved, improving the naturalness and realism of digital human voice interaction.

The invention relates to a learning evaluation method for English word pronunciation through audio recognition, which comprises the following steps: constructing a speech recognition system, preprocessing received user pronunciation audio, and then realizing phonetic symbol recognition of recognized words through vowel phonetic symbol recognition and screening and consonant phonetic symbol recognition and screening or full phoneme recognition. The word recognized by the final recognition phonetic symbol sequence obtained by the speech recognition system is compared with the stored standard pronunciation of the word, the pronunciation of the practicer is scored, and specific practice suggestions are given.

Techniques for enabling voice-based language learning are described. A system of the present disclosure supports receipt of speech inputs in a user's native language and a language the user is learning. The system runs an automatic speech recognition (ASR) component configured for transcribing speech in the native language (e.g., English) and runs a phoneme recognition component configured for recognizing phonemes of the learning language (e.g., Spanish). The system can respond to commands provided in the native language (e.g., “Stop lesson”, “Repeat lesson”, etc.) and provide pronunciation feedback with respect to spoken inputs in the learning language.

The invention relates to the technical field of data processing, and discloses a data calibration method and device, equipment and a medium, and the method comprises the steps: carrying out the phoneme recognition of a target audio through a target phoneme recognition model, obtaining the prediction phoneme and prediction probability of each audio frame, and obtaining the prediction probability of each audio frame according to a preset mapping table; determining a mapping phoneme of each sub-word in the transcription text corresponding to the target audio, for any audio frame, according to the predicted phoneme of the audio frame, the prediction probability corresponding to the predicted phoneme and the mapping phoneme of each sub-word, respectively matching the audio frame with each sub-word, and according to a matching result, determining a target sub-word corresponding to the audio frame, and according to the mapping phonemes of the target sub-words, the predicted phonemes of the audio frames are calibrated, and calibration results of the predicted phonemes of all the audio frames are fused to form a final calibration result. The method can be applied to the field of financial science and technology, the accuracy of phoneme recognition is improved, and a more reliable acoustic representation basis is provided for a downstream speech processing task of phoneme recognition.

The present disclosure relates to an audio analysis method, device, equipment and storage medium, and relates to the field of computer technology, and is used to solve the problem that general technologies have low accuracy and high cost when processing audio data of a terminal. The audio analysis method includes: obtaining dry audio data of a target audio; inputting the dry audio data into a target phoneme recognition model for phoneme recognition processing to obtain a target phoneme time series; the target phoneme recognition model is trained based on a phoneme set after homophonic merging of multiple languages; inputting the dry audio data into a target text recognition model for text recognition processing to obtain a target text time series; the target text recognition model is trained based on the lyrics information of the target audio; determining audio signal feature information of the dry audio data; and determining the audio analysis result of the dry audio data based on the target phoneme time series, the target text time series and the audio signal feature information.