51 results about "Viseme" patented technology

A computer implemented system is described in various embodiments herein, the system includes a processor, a memory coupled to the processor and storing processor-executable instructions that, when executed, configure the processor to: receive an initial video data object V comprising a plurality of image frame data objects I; receive a set of time-synchronized viseme parameters corresponding to target audio data object A′; and process the initial video data object V and the set of time-synchronized viseme parameters using a machine learning network to generate an output video data object V′, wherein initial mouth regions in the initial video data object V have been replaced with replacement mouth regions generated based on the set of time-synchronized viseme parameters.

The invention relates to the technical field of intelligent home control, and discloses a control method and a control system of an intelligent sound box. The control method of the intelligent sound box is applied to a control device, and specifically comprises the following steps: S101, collecting a voice instruction of a user, carrying out noise reduction processing and voiceprint feature extraction on the voice instruction, transmitting the processed voice instruction to a semantic analysis engine, carrying out intention recognition on voice content based on a deep learning model, and sending the intention recognition result to a semantic analysis engine; generating a structured operation instruction set; and S102, according to the device identifier contained in the structured operation instruction set. Through dual authentication of voiceprint feature extraction and dynamic verification phrase, the illegal access false recognition rate is reduced to below 0.12%, the voiceprint database is combined with the Mel frequency cepstrum coefficient and the dynamic time warping algorithm, accurate matching of user identities is achieved, the dynamically generated verification phrase comprises random initial and final combinations, and recording playback attacks are effectively resisted.

Methods and apparatus are described for evaluating dubbing of media content. Phonemes in dubbed audio are extracted and mapped to visemes. Lip poses in video frames of the media content corresponding to the phonemes of the dubbed audio are compared to the visemes determined from the dubbed audio. A notification may be generated based on the comparison that indicates synchronization of the dubbed audio to lip poses of the video.

A tone evaluation rehabilitation training device and system, the device comprises a flexible cap body and a bandage, the cap body is integrated with a 10-channel electrode plate to accurately cover language brain areas on both sides, electroencephalogram signals are transmitted in real time through Bluetooth 5.0, and a noise reduction earphone is used for providing sound output for a patient. The microphone is used for collecting audio of a patient and transmitting the audio to the processing module for analysis, the system dynamically regulates acoustic stimulation based on the neural activation feedback module, high-precision tone recognition is achieved in combination with audio preprocessing, SpecAugment data enhancement, a CNN-Transform mixed model and an attention mechanism, and training efficiency is optimized through pre-training model fine tuning and Warmup learning rate scheduling. An initial consonant-vowel-tone three-dimensional confusion probability table is introduced to divide interference intensity grades, high and low interference task paths are dynamically switched through a semantic error rate, and an evaluation module generates a four-tone accuracy rate, an F0 curve comparison graph and a personalized rehabilitation scheme. The recognition precision is improved, the training period of special crowds is shortened, and full-period self-adaptive rehabilitation is provided for preschool children to speech disorder patients.

One or more inputs are processed with a machine-learned language model to obtain a prediction output. The inputs comprise speech information descriptive of one or more first words spoken by a user, and the prediction output comprises one or more second words predicted to follow the one or more first words. A sequence of visemes formed to produce the one or more second words is determined. Based on the sequence of visemes, facial animation information is generated descriptive of a facial animation that animates a three-dimensional representation of a mouth of the user forming the sequence of visemes to speak the one or more second words.

The invention discloses a mandarin training correction method for voice processing, and relates to the technical field of voice signal processing, and the method comprises the steps: recording the voice when a user reads a training text, and the amplitude-frequency feature sequence of each read character voice; constructing corresponding vowel feature vectors and initial feature vectors; detecting non-standard vowel syllables of the user; detecting non-standard initial syllables of the user; and prompting the user, and broadcasting the corresponding standard mandarin syllable pronunciation and spelling method to the user so as to train and correct the mandarin of the user. According to the method, the specific syllables of the user are accurately captured and carefully recognized, the pronunciation and spelling problems of the user are effectively pointed out, mandarin training and correction of the user are more efficient and accurate, and the method has great significance in improving the language ability and communication level of the public.

This invention relates to artificial intelligence technology and discloses an intelligent question-answering method, apparatus, device, and storage medium based on nasal sounds. The method includes: splitting user-input speech audio data to obtain a phoneme sequence; converting the phoneme sequence into initials and finals according to nasal sound conversion rules to obtain a phoneme arrangement matrix, and quantizing it to obtain a phoneme vector matrix; performing a fully connected operation on each phoneme vector in the phoneme vector matrix to obtain a set of combined phoneme sequences; performing feature extraction and word segmentation recognition operations on each combined phoneme sequence in the combined phoneme sequence set to obtain a word segmentation arrangement matrix corresponding to each combined phoneme sequence; performing word segmentation and arrangement on each word segmentation arrangement matrix, identifying the completeness of the intent in the word segmentation and arrangement results to obtain a complete intent statement; and filtering the complete intent statement according to current business scenario information to obtain the speech intent text. This invention can increase the accuracy of speech recognition and improve the efficiency of intelligent question answering.

The invention discloses a semantic analysis method for voice in a crowded environment. The method comprises the following steps: acquiring voice signal data and lip movement video stream data of a target user; extracting a lip motion feature sequence, and mapping the lip motion feature sequence into a predicted speech feature vector; inputting a voiceprint separation model, separating a voice segment of a target user from the mixed voice signal, and generating pure voice data features; performing time domain segmentation on the pure voice data features to obtain a voice signal time domain waveform of a single character; matching the single character voice signal time domain waveform with an initial consonant and vowel time domain waveform library to obtain pinyin expression corresponding to each character; and carrying out tone combination correlation analysis on the pinyin expressions of the continuous single characters to obtain the meaning of the voice segment of the target user. The method has the advantages that the voice of the target user is effectively extracted by combining the lip movement video stream and the voice signal data and utilizing the deep learning and voiceprint separation technology, and the voice recognition accuracy in the noisy environment is remarkably improved.

This invention discloses a method for decoding Chinese character phrases and sentences based on stereotactic electroencephalogram (sEEG) signals. This method uses stereotactic electroencephalogram (sEEG) signals as system input to decode Chinese phonemes, achieving greater generalization and adaptability than syllables. After identifying the initials, finals, and tones that make up Chinese Pinyin (Pinyin has 23 initials and no initials, 24 finals, and five tones), this method performs a two-way correction process using a language model modeled by Bayesian probabilistic modeling and a large language model. The decoded and corrected phrases or sentences are then displayed in real time on a user interface.

The present invention relates to a method for processing Mandarin nasal voice signals, an experimental system and a method. The processing method includes the following steps: Step S1, collect Mandarin nasal voice signals, detect the vowel voice frames in the nasal voice signals, and perform classification detection on the vowel voice frames to determine whether the vowel voice frame belongs to one of the 16 nasal vowel voice frames containing n or ng, so as to detect the nasal vowel voice frames and determine the category of the nasal vowel voice frame, that is, determine which category of the 16 nasal vowel voice frames the nasal vowel voice frame belongs to; the above 16 nasal vowel voice frames containing n or ng refer to the voice frames of 8 front nasal vowels an, ian, uan, üan, en, uen, in, ün and the voice frames of 8 back nasal vowels ang, iang, uang, eng, ueng, ing, ong, iong; Step S2, perform nasal voice restoration processing on the nasal vowel voice frames, and do not perform processing on other voice frames. The present invention can achieve efficient processing of nasal voice.

The application discloses an input method, device and system based on number sequence and number sequence vowel definition and symbol mapping, which comprises the following steps: defining a number sequence and a number sequence vowel set, wherein the number sequence is a fixed sequence of numbers 1-9, 0 and an extended character, the number sequence vowel set comprises 11 groups of vowels arranged according to the number sequence, and each group of vowels corresponds to a unique number sequence coding bit; establishing a mapping relationship between the 11 groups of vowels and a number sequence input unit, wherein the number sequence input unit comprises at least 10 digital key positions and an extended character key position arranged according to the number sequence; and generating a candidate word list according to the defined number sequence and number sequence vowel set and the mapping relationship in response to a user input operation. The application solves the problem of inconsistent coding logic and multiple keystrokes of vowels in the prior art, significantly improves the input efficiency of Chinese pinyin, supports multiple input modes to share a unified coding system and reduces the learning cost of users.

The application discloses an input method, device and system based on number sequence final consonant definition and column direction mapping. The method comprises the following steps: according to the consistency principle of pronunciation tongue position, merging final consonants in the Chinese pinyin scheme into 11 groups of number sequence final consonants, establishing a column direction mapping relationship between the 11 groups of number sequence final consonants and 11 column character keys of a keyboard, so that the character final consonants on each column character key are merged into the corresponding number sequence final consonants through column direction compression; in response to the input operation of a user, converting the character final consonants into corresponding number sequence final consonant codes according to the mapping relationship, and generating a candidate word list. The application realizes single key triggering of final consonants by systematically merging final consonants into 11 groups and establishing a determined mapping with the column direction of the keyboard, solves the problems of multiple key strokes and inconsistent coding logic in the prior art, significantly improves the input efficiency of the Chinese pinyin, supports multiple input modes to share a unified coding system, and reduces the learning cost of users.

The invention discloses a method for decoding phrases and sentences of Chinese characters based on stereotactic electroencephalogram signals, which uses the stereotactic electroencephalogram (sEEG) signals as system input to decode Chinese phonemes so as to achieve generalization and suitability higher than those of syllables. According to the method, after initial consonants, finals and tones are recognized based on sEEG signals to form Chinese pinyin (23 initial consonants and no initial consonants of pinyin, 24 finals and five tones), two-way correction processing is conducted through a language model and a large language model of Bayesian probability modeling, and decoded and corrected phrases or sentences are displayed on a user interface in real time.

Provided are a method, apparatus, and medium for recognizing tones of speech. The method comprises: detecting the duration position of a vowel of a vowel in a speech to be recognized; detecting a tone core portion of the speech to be recognized based on the duration position of the vowel of the vowel; and identifying the tone category of the speech to be recognized based on the tone core portion. Thus, according to at least one embodiment of the present disclosure, the tone core portion can be more accurately detected within the duration position of the vowel of the vowel, thereby more accurately identifying the tone category.

The present disclosure relates to a mouth shape animation sequence generation method and device. The method comprises: processing input information including input audio and / or input text to obtain a phoneme sequence corresponding to the input information; determining a viseme sequence corresponding to each phoneme in the phoneme sequence based on the phoneme sequence and a matched target mapping table; adjusting the weight of each viseme in the viseme sequence according to the viseme sequence and the corresponding target information to form an adjusted viseme sequence; determining a mouth shape animation corresponding to each viseme in the adjusted viseme sequence based on the target mapping table; and generating a mouth shape animation sequence according to the mouth shape animation. The target mapping table matched with the input information is used to realize the generation of the phoneme sequence based on the input information, the viseme sequence, and the mouth shape animation sequence. The input requirement is low, the mouth shape animation sequence is suitable for different accents and languages, the generation speed is fast, the time is short, the synchronization matching degree with the audio is good, the accuracy is high, and manual debugging is not required, thereby reducing the labor cost.

The invention provides a pinyin input device, a pinyin input method and a computer program product, and the device comprises a keyboard module which is configured to have mapping of a first key and a key value; the initial consonant module is configured to have mapping of a second key and a key value, and the mapping of the second key and the key value is used for configuring the corresponding relation between the first letter group defined in the English letter set and the initial consonant key value group and the keys; the vowel module comprises a single vowel unit, a basic vowel unit and a derivative vowel unit; the syllable analysis module captures an input sequence generated by performing trigger operation on the keyboard, and analyzes the input sequence based on at least one of mapping of a second key and a key value configured in the initial consonant module and mapping of a third key and a key value, mapping of a fourth key and a key value and mapping of a fifth key and a key value configured in the final module; determining the initial consonant and vowel combination in the input sequence meeting the Chinese pinyin rule, and outputting words or phrases corresponding to the initial consonant and vowel combination.

The present disclosure provides a multi-functional speech recognition method, system and terminal. The present disclosure constructs an end-to-end speech recognition system with pinyin initial and final consonants as modeling units, and adds a modeling unit probability output module, effectively improving non-close sound character replacement errors, and relative to existing end-to-end speech recognition systems, adding various functions such as end-to-end speech recognition system acoustic recognition performance evaluation and pronunciation standard degree evaluation. The method comprises: receiving a speech to be recognized; performing acoustic feature extraction and encoding on the speech to be recognized; decoding the encoded acoustic features using a Chinese character decoder, wherein the Chinese character decoder uses pinyin initial and final consonants as modeling units, and maps the encoded acoustic feature sequence to a Chinese character sequence through initial and final consonants; and outputting a speech recognition result.

The application discloses a kind of personnel dense environment under the semantic analysis method of voice, comprising: acquisition obtains the voice signal data of target user, lip movement video stream data;Extract lip movement feature sequence, map lip movement feature sequence into predicted voice feature vector;Input voiceprint separation model, separate the voice segment of target user from mixed voice signal, generate pure voice data characteristics;Pure voice data characteristics are carried out time domain segmentation, obtain the phonetic time domain waveform of single word;Single word phonetic time domain waveform is matched with initial, final time domain waveform library, obtains the corresponding pinyin expression of each word;Tone combination correlation analysis is carried out to the pinyin expression of continuous single word, obtains the meaning of the voice segment of target user.The application has the advantages that: by combining lip movement video stream and voice signal data, using deep learning and voiceprint separation technology, the voice of target user is effectively extracted, and the voice recognition accuracy in noisy environment is significantly improved.

The invention provides a method and a system for converting natural intonation into melody, a terminal and a medium, which are characterized in that fundamental frequency and tone length information of a voice sample are extracted, initial and final consonants are distinguished, fundamental frequency parameters of each final consonant are calculated, and the tone tuning range of each final consonant in a statement is determined. And matching music symbols by using the tuning ranges to generate a preliminary melody, and finally aligning the melody with the syllables of the voice sample to generate an audio file with a song melody. On the basis of the phonetic theory, the fundamental frequency, the tone length and the pause interval information of the voice sample are accurately extracted, and it is ensured that the generated melody can accurately reflect the characteristics and rhythm of natural intonations. The automatic method not only simplifies the operation process and improves the conversion efficiency, but also is suitable for a plurality of fields such as linguistics research, speech synthesis, language teaching, language training and pronunciation improvement.

The invention discloses a phonetic and morphological code Chinese character input method based on binary syllabification and initial consonants of head and tail components, belongs to the technical field of Chinese character input methods, and aims at solving the problems that an existing phonetic and morphological code input method is unreasonable in key position distribution, high in repeated code rate and too long in code length. The final key positions are mnemonic with the aid of final poems, are classified and distributed at 30 key positions according to final heads, and are combined with compound finals and the like; initial keys correspond to a standard English keyboard, V = zh, I = ch, U = sh, O is a zero initial, y and w are regarded as independent initial, and special keys reuse vowel keys. The initial consonants of radicals or strokes are taken by the head and tail parts, 201 radicals are adopted as the radicals, strokes are taken by non-radicals, the initial consonants of the radicals are taken by the head parts when a single character is the radicals, keys of part of the parts are adjusted, and auxiliary memorization is carried out by marking additional keys through visual elements. Common character codes are'initial consonants + final consonants + initial consonants of a first part + initial consonants of a tail part ', and the codes of words adopt different rules. The method has the advantages that no repeated code is added; the learning cost is low; the memory burden is relieved; the input efficiency is improved; and the repeated code rate is reduced.

The invention relates to the technical field of smart home, and discloses an electrical equipment control method and device, electrical equipment, a medium and a product. Static initial consonant features and final features are obtained by performing feature extraction on dialect voice information of a user, and dynamic transition features of user pronunciation from initial consonants to final vowels are obtained; and capturing the difference of different dialects in the aspects of initial and final connection and tone change. Therefore, the accuracy of dialect recognition is improved based on the initial consonant features, the vowel features and the transition features. And then, the dialect instruction set corresponding to the dialect type is utilized to convert the dialect instruction of the user into a standard instruction, so that the user can control the electrical equipment through the dialect voice, and the use experience of the user is improved.

The present application relates to the technical field of speech recognition, and aims to solve the problem of lack of training samples of heavy accent speech recognition model. A method, device and computer equipment for generating a sample set for training a speech recognition model are provided, wherein the method comprises: decoding a target command word into a toneless original pinyin sequence; based on a heavy accent rule library (heavy accent refers to non-standard pronunciation of initial / final change) constructed according to common non-standard pronunciation rules, a heavy accent pinyin sequence is generated through coding conversion; a heavy accent audio is generated through a text-to-speech audio generation tool; an audio input is input into a preset recognition model to obtain a recognition result and convert it into a recognized pinyin sequence; a screening rule is constructed based on the original and / or heavy accent pinyin, and the recognized pinyin sequence is compared to screen the audio; and the audio meeting the requirements is collected to form a training sample set. Through rule-based generation and accurate screening, high-quality heavy accent samples can be efficiently obtained, and the heavy accent recognition performance of the model can be improved.

This disclosure provides a method, apparatus, device, and storage medium for segmenting Chinese speech signals. The method includes: sampling a target audio signal containing speech corresponding to a target Chinese text to obtain signal amplitudes corresponding to multiple sampling points; performing speech endpoint detection on the target audio data based on the signal amplitudes corresponding to the multiple sampling points to obtain multiple speech segments in the target audio data; determining the vowel position sequence corresponding to the target speech segment based on the formant energy of the speech signal in the target speech segment; determining syllable segmentation points and initial / final segmentation points of the target speech segment based on the signal amplitudes of sampling points between two adjacent vowel positions in the vowel position sequence and the initials and finals of the corresponding Chinese text segment; and segmenting the target speech segment based on the syllable segmentation points and initial / final segmentation points to obtain segmented speech primitives. This disclosure improves the accuracy of speech signal segmentation.

The application belongs to the technical field of lip language mouth shape analysis, and discloses a GNN-LSTM Chinese lip language classification method based on node multi-association graph information fusion. Lip key points are represented by three structures of adjacency graph, symmetric graph and upper and lower lip relationship graph. High-dimensional space-time features are extracted under the synergistic effect of graph convolutional neural network and long short-term memory network, the space-time global correlation between lip key points is effectively captured, and the mouth shape class is divided based on initial and final vowels. The multi-level collaborative relationship between initial and final vowels is considered, the influence of mouth shape similarity and visual ambiguity on model performance is reduced, a mouth shape library is established by using the extracted high-dimensional space-time features, the lip shape and corresponding pinyin are more discriminatively mapped and induced, the influence of mouth shape similarity and visual ambiguity on model performance is reduced, the method can adapt to complex changes and many-to-one mapping phenomena in actual pronunciation processes, and effectively enhances the accuracy and robustness of subsequent mouth shape classification.

A system and method of modulating animation curves based on audio input. The method including: identifying phonetic features for a plurality of visemes in the audio input; determining viseme animation curves based on parameters representing a spatial appearance of the plurality of visemes; modulating the viseme animation curves based on melodic accent, pitch sensitivity, or both, based on the phonetic features; and outputting the modulated animation curves.

The present invention discloses a Chinese character Rubik's Cube, which includes a three-dimensional dictionary. On the front of the three-dimensional dictionary, initials are arranged longitudinally and finals are arranged horizontally. A character nest is opened at the intersection of each initial and final, and a character rod is arranged in each character nest. The front of the character rod is provided with the pinyin after the corresponding initial and final are combined, forming a character library of homophonic and homographic characters. A number of sound nests are arranged on the front of the three-dimensional dictionary, and sound rods are arranged in the sound nests. A placement groove is opened on the front of the three-dimensional dictionary, and a printing device is placed in the placement groove. By utilizing the characteristics of single-syllable movable Chinese characters, the present invention changes the linear arrangement of the existing dictionary according to the first letter to a three-dimensional dictionary with a vertical and horizontal arrangement composed of initials and finals, which is conducive to learners quickly looking up characters, recognizing and learning to use Chinese characters regularly. At the same time, Chinese characters are in one-to-one correspondence with foreign language words, which is conducive to bilingual learning and also facilitates spelling and storage.

The embodiment of the invention provides a voiceprint recognition method and device, electronic equipment and a storage medium, belongs to the technical field of voiceprint recognition, and is suitable for the fields of financial science and technology and medical treatment. The method comprises the following steps: acquiring voiceprint verification voice and voiceprint registration voice of a target user; performing voice recognition on the voiceprint verification voice to obtain a voiceprint verification text; based on the voiceprint verification text, performing vowel labeling on the voiceprint verification voice to obtain a verification voice vowel label; performing vowel coverage rate calculation on the voiceprint verification voice based on the verification voice vowel label and a preset total number of basic vowels to obtain a verification voice vowel coverage rate; performing vowel distribution feature extraction on the voiceprint verification voice based on the verification voice vowel label to obtain verification voice vowel distribution; and performing voiceprint recognition on the target user based on the voiceprint registration voice, the total number of basic vowels, the verification voice vowel coverage rate and the verification voice vowel distribution characteristics. According to the embodiment of the invention, the voiceprint recognition efficiency can be improved.

The invention belongs to the technical field of information processing, and relates to a phonetic code and numerical code combined Chinese and English input method, which comprises an English input method and a Chinese input method, and is characterized in that English letters are coded, brevity codes are set, and deletion codes of corresponding letters of two bits at the head and the tail of an English word are used for quick retrieval and correspondence; when multiple repeated code words exist, complement codes are matched for further screening, initial consonants and finals of Chinese pinyin are coded, high-frequency representative characters are coded and then used for replacing same-syllable characters, complicated English and pinyin input is converted into digital input, the Chinese and English input accuracy and efficiency of intelligent equipment are improved, the repeated code rate is reduced, only numeric keys are needed for input, and the input efficiency is improved. The method is convenient and fast, reduces typewriting difficulty, improves Chinese character input accuracy and efficiency of a computer, facilitates operation under the conditions of virtual keyboards and entity keypads of smart phones and information processing equipment, has certain rules in sorting and coding, is easy to learn, is suitable for users of different age groups, and is easy to popularize on a large scale.

The present disclosure provides a rhyming word recommendation method and device, equipment and storage medium, relates to the technical field of computers, and particularly relates to the technical field of intelligent recommendation. The specific implementation scheme is as follows: obtaining a to-be-matched rhyming word and a rhyming configuration parameter; determining a target foot of a sentence to which the to-be-matched rhyming word belongs according to the to-be-matched rhyming word; determining a target rhyming word class corresponding to the target foot in a rhyming word class included in a rhyming word library, the feet of the sentences to which the rhyming words in each rhyming word class belong are identical in tone; determining a rhyming word matched with the to-be-matched rhyming word from the target rhyming word class according to the rhyming configuration parameter; and recommending the determined rhyming word. The scheme provided in the present disclosure can realize rhyming word recommendation.