Information mapping method, device, storage medium and electronic device
By splitting the pinyin of Chinese characters and performing lip-like decomposition and mapping, the problem of low matching between virtual digital lip-like and text pronunciation is solved, and the authenticity and user experience of virtual digital human explanations are improved.
Patent Information
- Application Number
- CN202110550495.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-20
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-05-20
AI Technical Summary
The prior art is difficult to maximize the matching of the lip shape of a virtual digital person with the real pronunciation of the text, resulting in low synchronization between the lip shape and pronunciation.
By splitting the pinyin of Chinese characters into separate finals or initials + finals, the pronunciation lip type is determined to be a single mouth shape or a compound mouth shape, and the lip type is decomposed and mapped according to the mapping relationship between phoneme set and mouth shape. Improve the matching of the lip shape of the virtual object and the pronunciation of the text.
It achieves a higher degree of matching between the lip shape of the virtual object and the pronunciation of the text, and improves the authenticity and user experience of the virtual digital human explanation.
Smart Images

Figure CN113284506B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of human-computer interaction technology, and in particular to an information mapping method, device, readable storage medium and electronic device. Background Art
[0002] Virtual digital people refer to virtual characters with digital appearance. Virtual digital people rely on display devices to exist, have the appearance of real people or cartoon people, have human behavior, and have the ability to express themselves through language, facial expressions and body movements. In recent years, the virtual digital human industry has developed vigorously, especially in the e-commerce live broadcast industry. For example, after the anchor inputs the introduction text for a certain product, the virtual digital person on the screen explains it according to the text. How to maximize the match between the virtual digital person's lip shape and the real pronunciation of the text is a problem that needs to be solved urgently. Summary of the invention
[0003] The embodiments of the present invention provide an information mapping method, device, readable storage medium and electronic device to improve the matching degree between the lip shape of a virtual object and the real pronunciation of a text.
[0004] The technical solution of the embodiment of the present invention is achieved as follows:
[0005] An information mapping method, the method comprising:
[0006] Splitting the pinyin of the first Chinese character into: a single first vowel, or: a first consonant + a second vowel, wherein the first vowel and the second vowel are: a single vowel or multiple vowels; wherein the first Chinese character is any Chinese character that the virtual object is going to pronounce;
[0007] According to the mouth shape change characteristics of the first vowel or the second vowel during pronunciation, the pronunciation mouth shape type of the first vowel or the second vowel is determined, wherein the pronunciation mouth shape type includes: a single mouth shape and a compound mouth shape;
[0008] If the result of the judgment is a compound mouth shape, the mouth shape of the first vowel or the second vowel is decomposed;
[0009] According to the mouth shape decomposition result, the pronunciation mouth shape of the first vowel or the first consonant + the second vowel is mapped to the virtual object.
[0010] The method of determining the pronunciation mouth shape type of the first vowel or the second vowel comprises:
[0011] If the shape of the mouth does not change when the first vowel or the second vowel is pronounced, the pronunciation mouth shape type of the first vowel or the second vowel is determined to be a single mouth shape; otherwise, the pronunciation mouth shape type of the first vowel or the second vowel is determined to be a compound mouth shape.
[0012] The method of determining the pronunciation mouth shape type of the first vowel or the second vowel comprises:
[0013] If the first vowel or the second vowel is one of the following, the pronunciation mouth shape type of the first vowel or the second vowel is single mouth shape:
[0014] a, o, e, i, u, ü, ang, eng, ing, ong, ai, ei, ao, ou, iu, ie, üe, er, an, en, in, ün, zhi, chi, shi, ri, zi, ci, si, wu, yi, yu, ye, yue, yin, yun, ying;
[0015] If the first vowel or the second vowel is one of the following, the pronunciation mouth shape type of the first vowel or the second vowel is a compound mouth shape:
[0016] ian, iao, iang, iong, uan, üan, uai, uang, ui, un, uo, ia, yuan.
[0017] The method further comprises:
[0018] According to the changing rules of muscles and mouth shapes when pronouncing each initial consonant and each final, phonemes with the same mouth shape changes are grouped into a set, and a mapping relationship between the phoneme set and the mouth shape is established, wherein the phonemes are initial consonants, single finals or multiple finals;
[0019] The step of performing mouth shape decomposition on the first vowel or the second vowel comprises:
[0020] According to the mapping relationship between the phoneme set and the mouth shape, the first final or the second final is decomposed into a plurality of phonemes, and the mouth shape corresponding to each decomposed phoneme is obtained.
[0021] The method of grouping the phonemes with the same mouth shape changes into a set and establishing a mapping relationship between the phoneme set and the mouth shape includes:
[0022] Phoneme set 1: {a, er, an, ang, iang, ia, ai, an, ian}, corresponding to the mouth shape: open mouth with a large opening range;
[0023] Phoneme set 2: {o, ou, ong}, corresponding mouth shapes: round mouth shape;
[0024] Phoneme set three: {e}, corresponding to the mouth shape: mouth relaxed, mainly relying on the throat to pronounce;
[0025] Phoneme set 4: {d, t, n, l, g, k, h}, corresponding to the following mouth shapes: slightly open mouth, mainly relying on the tongue for pronunciation;
[0026] Phoneme set 5: {i, in, ing, y, ie, zh, ch, sh, r, z, c, s, j, q, x}, corresponding to the mouth shape: mouth slightly split backwards, mouth shape flat;
[0027] Phoneme set 6: {ei, en, eng}, the corresponding mouth shape is: open mouth, and the mouth opening amplitude is smaller than that of phoneme set 1;
[0028] Phoneme set seven: {w, u, ü, üe, un, iu, iong}, corresponding mouth shape: mouth shape gathered;
[0029] Phoneme set eight: {ao, iao}, corresponding to the mouth shape: the mouth shape is oval, and the lower lip is round;
[0030] Phoneme set nine: {b, p, m}, corresponding mouth shape: mouth basically closed.
[0031] After obtaining the mouth shape corresponding to each decomposed phoneme and before mapping the pronunciation mouth shape of the first vowel or the first consonant + the second vowel to the virtual object, the method further includes:
[0032] For the first initial consonant + the second final, determine whether the mouth shape of the first initial consonant is not obvious in the combined pronunciation based on the changes in the mouth shape of the combined pronunciation of the first initial consonant and the second final. If so, ignore the mouth shape of the first initial consonant and determine to map only the mouth shape of each phoneme decomposed from the second final to the virtual object.
[0033] The step of judging whether the mouth shape of the first initial consonant is not obvious in the combined pronunciation according to the mouth shape change of the combined pronunciation of the first initial consonant and the second final consonant comprises:
[0034] Determine whether the first initial consonant + the second final is the following combination. If so, determine that the pronunciation mouth shape of the first initial consonant is not obvious in the combined pronunciation:
[0035] duan, tuan, nuan, luan, guan, kuan, huan, guai, huai, kuai, guang, kuang, huang, juan, guo, kuo, huo, luo, tuo, yuan.
[0036] After determining the pronunciation mouth shape type of the first vowel or the second vowel, the method further comprises:
[0037] If the judgment result is a single mouth shape, and the first initial consonant + the second final is the following combination, then the pronunciation of the first initial consonant is ignored, and only the pronunciation mouth shape of the second final is mapped to the virtual object:
[0038] lü, wu, yu, yi, yin, ying.
[0039] The step of performing mouth shape decomposition on the first vowel or the second vowel comprises:
[0040] If the first or second final is ui, the pronunciation of ui is decomposed into the mouth shape of u and the mouth shape of ei;
[0041] If the first or second final is un, the pronunciation mouth shape of un is decomposed into the mouth shape of u and the mouth shape of en.
[0042] The method of splitting the pinyin of the first Chinese character into: the first initial consonant + the second final and before determining the pronunciation mouth shape type of the second final further includes:
[0043] If the combination of the first initial consonant + the second final is yun, then the pronunciation of y and n is ignored, and only the pronunciation of u is mapped to the virtual object;
[0044] If the combination of the first initial consonant + the second final is ye, then the pronunciation of y is ignored, and the pronunciation mouth shape of e is replaced by the pronunciation mouth shape of i, and the pronunciation mouth shape of i is mapped to the virtual object as the pronunciation mouth shape of ye;
[0045] If the combination of the first initial consonant + the second final is yue, then the pronunciation of y and e is ignored, and only the pronunciation of u is mapped to the virtual object;
[0046] If the combination of the first initial consonant + the second final is zhi, chi, shi, ri, zi, ci or si, only the pronunciation mouth shape of i is mapped to the virtual object.
[0047] An information mapping device, the device comprising:
[0048] A pinyin splitting module is used to split the pinyin of the first Chinese character into: a single first vowel, or: a first consonant + a second vowel, wherein the first vowel and the second vowel are: a single vowel or multiple vowels; wherein the first Chinese character is any Chinese character that the virtual object is going to pronounce;
[0049] The mouth shape decomposition module is used to determine the mouth shape type of the first vowel or the second vowel according to the mouth shape change characteristics of the first vowel or the second vowel when pronouncing the first vowel or the second vowel, wherein the mouth shape type includes: a single mouth shape and a compound mouth shape; if the judgment result is a compound mouth shape, the mouth shape of the first vowel or the second vowel is decomposed;
[0050] The mouth shape mapping module is used to map the pronunciation mouth shape of the first vowel or the first consonant + the second vowel to the virtual object according to the mouth shape decomposition result.
[0051] A non-transitory computer-readable storage medium stores instructions, which, when executed by a processor, cause the processor to perform the steps of any of the above information mapping methods.
[0052] An electronic device includes the non-transitory computer-readable storage medium as described above, and the processor capable of accessing the non-transitory computer-readable storage medium.
[0053] In the embodiment of the present invention, according to the characteristics of the mouth shape changes of the finals in the pinyin of the Chinese character when pronouncing the finals, it is determined whether the pronunciation mouth shape of the finals is a single mouth shape or a compound mouth shape; if it is a compound mouth shape, the finals are decomposed into mouth shapes, and according to the mouth shape decomposition result, the pronunciation mouth shape of the pinyin of the Chinese character is mapped to the virtual object, thereby maximizing the matching degree between the mouth shape of the virtual object and the real pronunciation of the text. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0055] Figure 1 A flow chart of an information mapping method provided by an embodiment of the present invention;
[0056] Figure 2 A flow chart of an information mapping method provided by another embodiment of the present invention;
[0057] Figure 3 A schematic diagram of the structure of an information mapping device provided by an embodiment of the present invention;
[0058] Figure 4 An exemplary structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0060] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.
[0061] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0062] The embodiment of the present invention provides an information mapping method, in which the pinyin of a first Chinese character is split into: a single first vowel, or: a first initial consonant + a second vowel, wherein the first vowel and the second vowel are: a single vowel or multiple vowels, and the first Chinese character is any Chinese character that a virtual object will pronounce; according to the mouth shape change characteristics of the first vowel or the second vowel during pronunciation, the pronunciation mouth shape type of the first vowel or the second vowel is judged, and the pronunciation mouth shape types include: a single mouth shape and a compound mouth shape; if the judgment result is a compound mouth shape, the first vowel or the second vowel is decomposed into a mouth shape; according to the mouth shape decomposition result, the pronunciation mouth shape of the first vowel or the first initial consonant + the second vowel is mapped to the virtual object. The embodiment of the present invention determines whether the pronunciation mouth shape type of the final in the pinyin of the Chinese character is a single mouth shape or a compound mouth shape according to the characteristics of the mouth shape changes when pronouncing the final; if it is a compound mouth shape, the final is decomposed into a mouth shape, and according to the mouth shape decomposition result, the pronunciation mouth shape of the pinyin of the Chinese character is mapped to a virtual object, thereby maximizing the matching degree between the mouth shape of the virtual object and the real pronunciation of the text.
[0063] Figure 1 The information mapping method flow chart provided in one embodiment of the present invention has the following specific steps:
[0064] Step 101: Split the pinyin of the first Chinese character into: a single first vowel, or: a first consonant + a second vowel, wherein the first vowel and the second vowel are: a single vowel or multiple vowels; wherein the first Chinese character is any Chinese character that the virtual object is to pronounce.
[0065] Single vowels include: a, o, e, i, u, ü, ang, eng, ing, ong, ai, ei, ui, ao, ou, iu, ie, üe, er, an, en, in, un, ün. Poly-vowels are the finals composed of multiple single vowels, such as: ian, iao, iang, iong, uan, üan, uai, uang, uo, ia, etc.
[0066] Step 102: judging the pronunciation mouth shape type of the first final or the second final according to the mouth shape change characteristics when pronouncing the first final or the second final, where the pronunciation mouth shape types include: a single mouth shape and a compound mouth shape.
[0067] Step 103: If the determination result is a compound mouth shape, the mouth shape of the first final or the second final is decomposed.
[0068] Step 104: Map the pronunciation mouth shape of the first vowel or the first initial consonant + the second vowel to the virtual object according to the mouth shape decomposition result.
[0069] In the above embodiment, according to the characteristics of the mouth shape changes when pronouncing the finals in the pinyin of the Chinese character, it is determined whether the pronunciation mouth shape type of the finals is a single mouth shape or a compound mouth shape; if it is a compound mouth shape, the finals are decomposed into mouth shapes, and according to the mouth shape decomposition result, the pronunciation mouth shape of the pinyin of the Chinese character is mapped to the virtual object, thereby maximizing the matching degree between the mouth shape of the virtual object and the real pronunciation of the text.
[0070] The virtual object in the embodiment of the present invention refers to a virtual object with a digital appearance that can simulate pronunciation, such as a virtual digital person.
[0071] In an optional embodiment, if the shape of the mouth does not change when pronouncing the first final or the second final, the pronunciation mouth shape type of the first final or the second final is determined to be a single mouth shape; otherwise, the pronunciation mouth shape type of the first final or the second final is determined to be a compound mouth shape.
[0072] In an optional embodiment, in step 102, determining the pronunciation mouth shape type of the first final or the second final includes:
[0073] If the first vowel or the second vowel is one of the following, the pronunciation mouth shape type of the first vowel or the second vowel is single mouth shape:
[0074] a, o, e, i, u, ü, ang, eng, ing, ong, ai, ei, ao, ou, iu, ie, üe, er, an, en, in, ün, zhi, chi, shi, ri, zi, ci, si, wu, yi, yu, ye, yue, yin, yun, ying;
[0075] If the first vowel or the second vowel is one of the following, the pronunciation mouth shape type of the first vowel or the second vowel is a compound mouth shape:
[0076] ian, iao, iang, iong, uan, üan, uai, uang, ui, un, uo, ia, yuan.
[0077] At present, the basic pronunciation list of Chinese mouth shape is listed in the <<Hanyu Pinyin Scheme>>. There are 32 phonemes in Chinese Pinyin, which are divided into two categories: vowels and consonants: 10 vowels and 22 consonants. Among them, the 10 vowels include: a, o, e, ê, i, u, ü, -i (front), -i (back), er; the 22 consonants include: b, p, m, f, d, t, n, l, k, g, h, j, q, x, zh, ch, sh, r, z, c, s, ng. Among them, ê, -i (front), -i (back), and ng are not commonly used in Pinyin. The remaining 28 consonants and vowels are roughly divided into 3 categories according to the changes in the mouth shape, as shown in Table 1:
[0078]
[0079] Table 1
[0080] In the lip animation synthesis of virtual objects, the lip shape is made according to the pronunciation method. If a corresponding lip shape is made for each pronunciation, it will not only waste time but also waste memory space. Therefore, the embodiment of the present invention reclassifies the phonemes according to the lip shape description described in Table 1, and fully considers the pronunciation of compound phonemes and maps them to the corresponding lip shape. The embodiment of the present invention plans a new mapping relationship between phonemes and pronunciation lip shapes by performing statistical analysis on pinyin classification. Among them, phoneme is the smallest unit of pronunciation.
[0081] Chinese Pinyin consists of a single vowel, or an initial consonant + a vowel. There are 23 initial consonants and 32 finals, including single vowels (a, o, e, etc.), double vowels (an, ao, etc.) and compound vowels (ian, iao, etc.), and 16 whole-word syllables. In the embodiment of the present invention, all initial consonants, finals and whole-word syllables are classified according to the pronunciation mouth shape, and the mapping relationship is shown in Table 2:
[0082]
[0083] Table 2
[0084] Figure 2 A flowchart of an information mapping method provided by another embodiment of the present invention, wherein the specific steps are as follows:
[0085] Step 201: According to the changing rules of muscles and mouth shape when pronouncing each initial consonant and each final, phonemes with the same mouth shape change are grouped into a set, and a mapping relationship between the phoneme set and the mouth shape is established, wherein the phoneme is an initial consonant, a single final or multiple finals.
[0086] The embodiment of the present invention mainly considers that people have inevitable changes in mouth shape when speaking certain Chinese characters, and needs to set voice visemes; considering that the resources of the model are not easy to be too large, it is not possible to set the pronunciation mouth shape for each phoneme separately. Therefore, according to the changing rules of muscles and mouth shape when pronouncing each timbre, the phonemes with the same mouth shape change are classified into one category, that is, according to the changing rules of muscles and mouth shape when pronouncing each initial consonant and each final vowel, the phonemes with the same mouth shape change are classified into a set, and a mapping relationship between the phoneme set and the mouth shape is established, wherein the phoneme is an initial consonant, a single final or multiple finals. Table 3 shows the mapping relationship between each phoneme set and the pronunciation mouth shape, and the several pronunciation mouth shapes shown in Table 3 represent all the pronunciation mouth shapes. Among them, the mouth shape of phoneme set four can be changed from the mouth shape of phoneme set three, and the mouth shape of phoneme set six can be changed from the mouth shape of phoneme set one. Therefore, seven mouth shapes are finally used to represent the pronunciation of all Chinese characters, which is very conducive to rendering the mouth shape of virtual objects on the terminal.
[0087]
[0088]
[0089] Table 3
[0090] In Table 3, some compound finals, such as ia, ian, iang, and iong, belong to compound mouth shape finals in the mouth shape analysis. They need to read i first and then read a or u mouth shape. However, in the real pronunciation, the pronunciation mouth shape of i is not obvious. Therefore, when designing the mouth shape of the virtual object, the middle i should not have mouth shape changes. The specific explanation is shown in Table 4:
[0091]
[0092] Table 4
[0093] Step 202: Split the pinyin of the first Chinese character into: a single first vowel, or: a first consonant + a second vowel, wherein the first vowel and the second vowel are: a single vowel or multiple vowels; wherein the first Chinese character is any Chinese character that the virtual object is going to pronounce.
[0094] Step 203: judging the pronunciation mouth shape type of the first final or the second final according to the mouth shape change characteristics when pronouncing the first final or the second final, where the pronunciation mouth shape types include: a single mouth shape and a compound mouth shape.
[0095] Specifically, the first vowel or the second vowel is searched in Table 2. After the first vowel or the second vowel is found, it is checked in Table 2 whether the pronunciation mouth shape type corresponding to the first vowel or the second vowel is a single mouth shape or a compound mouth shape.
[0096] If the pronunciation mouth shape type of the first vowel or the second vowel is a single mouth shape, the pronunciation mouth shape of the first vowel, or the pronunciation mouth shape of the first initial consonant + the pronunciation mouth shape of the second vowel is directly mapped to the virtual object.
[0097] Step 204: If the determination result is a complex mouth shape, the first vowel or the second vowel is decomposed into a plurality of phonemes according to the mapping relationship between the phoneme set and the mouth shape, and the mouth shape corresponding to each decomposed phoneme is obtained.
[0098] Specifically, according to the phonemes in Table 3, the first final or the second final is decomposed into a plurality of phonemes in Table 3, and for each decomposed phoneme, its corresponding pronunciation mouth shape is searched in Table 3.
[0099] Step 205: According to the mouth shape decomposition result, the pronunciation mouth shape of the first vowel or the first initial consonant + the second vowel is mapped to the virtual object.
[0100] In the above embodiment, according to the changing rules of muscles and mouth shapes when pronouncing each initial consonant and each final, phonemes with the same mouth shape changes are grouped into a set, and a mapping relationship between the phoneme set and the mouth shape is established. When the final of the pinyin of the Chinese character is a compound mouth shape, according to the mapping relationship between the phoneme set and the mouth shape, the first final or the second final is decomposed into multiple phonemes, and the mouth shape corresponding to each decomposed phoneme is obtained. According to the mouth shape decomposition result, the pronunciation mouth shape of the first final or the first initial consonant + the second final is mapped to the virtual object, thereby eliminating the need to set a mouth shape for each phoneme, reducing the number of mouth shapes, and maximizing the matching degree between the mouth shape of the virtual object and the real pronunciation of the text.
[0101] In the embodiment of the present invention, when mapping the pinyin of a Chinese character to a pronunciation mouth shape, the pronunciation tone (i.e., tone) of the phoneme is not considered, because the pronunciation mouth shapes of all different tones of the same pinyin embodied in the virtual object are the same, and therefore the pronunciation tone is not considered. Table 5 lists all the pinyins of all Chinese characters, including the unstressed pronunciation of the finals and the combined pronunciation of the finals and the initials, wherein the capital letters are the pinyins of the Chinese characters, and the lowercase letters in brackets are the individual first finals or the first initials + the second finals obtained after the pinyin is split:
[0102]
[0103]
[0104] Table 5
[0105] The following are examples of Chinese characters corresponding to all pinyins, where each pinyin corresponds to one Chinese character regardless of pronunciation and tone:
[0106] : Ah, An'ang, Dad helps with the day shift. It's collapsed and changed. The table is different. Bin Bing broadcasts. Don't wipe the restaurant warehouse. Cao Ce Cen rubs tea. The firewood production field overtakes. Chen Cheng eats insects and is ugly. In addition to blowing through the window, spring pokes this from the gathering. The rough and crisp storage is wrong. The big generation is in power. The must wait for the ground to be coquettish. The electric adjustment falls and loses. All the movements are Du Duan. Hungry, eh, and the hair is released. Is the Buddha blessed? The brother gives the stem work. The valley hangs the strange pipe. The light track rolls , The owl is old and tired, and the two of them are chatting in a row. There are six dragons in Linling, and the reed eggs are lost. The green mother is full of buys, and the sisters are busy with fur. The cute rice noodles are meowing, and the Minming Miu Mo Mou is Mu Na milk is difficult to sac. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. , The first time, and the last. ,
[0107] By reading the pronunciation of each Chinese character, it is found that some initials are mainly pronounced by the tongue or throat, so there is no need to set the mouth shape. By testing and verifying each phoneme combination, it is found that for diphthong phonemes such as uan, speech recognition will give the phoneme uan, because uan is composed of the pronunciation mouth shapes of u and an. Therefore, the present invention divides uan into speech viseme u and speech viseme a. In speech recognition, the overall pronunciation of the phoneme uan is given, and the time axis is start_time and end_time. In order to ensure the synchronization of the mouth shape time axis, considering that the duration of the middle phoneme u is shorter and the duration of an is longer, the duration of the uan phoneme is set to duration = end_time-start_time, then the start time of u is start_time, and the end time is 0.3*duration; the start time of an is start_time+0.3*duration, and the end time is end_time. According to this pronunciation rule, all pronunciation types are counted and the following pronunciation rules are set:
[0108] In an optional embodiment, after obtaining the mouth shape corresponding to each decomposed phoneme in step 204 and before mapping the pronunciation mouth shape of the first vowel or the first initial consonant + the second vowel to the virtual object in step 205, the method further includes:
[0109] For the first initial consonant + the second final, determine whether the mouth shape of the first initial consonant is not obvious in the combined pronunciation based on the changes in the mouth shape of the combined pronunciation of the first initial consonant and the second final. If so, ignore the mouth shape of the first initial consonant and determine to map only the mouth shape of each phoneme decomposed from the second final to the virtual object.
[0110] In an optional embodiment, judging whether the mouth shape of the first initial consonant is not obvious in the combined pronunciation according to the mouth shape change of the combined pronunciation of the first initial consonant and the second final includes:
[0111] Determine whether the first initial consonant + the second final is the following combination. If so, determine that the pronunciation mouth shape of the first initial consonant is not obvious in the combined pronunciation:
[0112] duan, tuan, nuan, luan, guan, kuan, huan, guai, huai, kuai, guang, kuang, huang, juan, guo, kuo, huo, luo, tuo, yuan.
[0113] In an optional embodiment, in step 103, performing mouth shape decomposition on the first final or the second final includes:
[0114] If the first or second final is ui, the pronunciation of ui is decomposed into the mouth shape of u and the mouth shape of ei;
[0115] If the first or second final is un, the pronunciation mouth shape of un is decomposed into the mouth shape of u and the mouth shape of en.
[0116] In an optional embodiment, in step 102, after determining the pronunciation mouth shape type of the first final or the second final, it further includes: if the determination result is a single mouth shape, and the first initial consonant + the second final is the following combination, then it is determined to ignore the pronunciation of the first initial consonant, and only map the pronunciation mouth shape of the second final to the virtual object: lü, wu, yu, yin, ying.
[0117] In an optional embodiment, in step 101, after splitting the pinyin of the first Chinese character into: the first initial consonant + the second final and before determining the pronunciation mouth shape type of the second final, further includes:
[0118] If the combination of the first initial consonant + the second final is yun, then the pronunciation of y and n is ignored, and only the pronunciation of u is mapped to the virtual object;
[0119] If the combination of the first initial consonant + the second final is ye, then the pronunciation of y is ignored, and the pronunciation mouth shape of e is replaced by the pronunciation mouth shape of i, and the pronunciation mouth shape of i is mapped to the virtual object as the pronunciation mouth shape of ye;
[0120] If the combination of the first initial consonant + the second final is yue, then the pronunciation of y and e is ignored, and only the pronunciation of u is mapped to the virtual object;
[0121] If the combination of the first initial consonant + the second final is zhi, chi, shi, ri, zi, ci or si, only the pronunciation mouth shape of i is mapped to the virtual object.
[0122] The above-mentioned embodiments utilize the real characteristics of Chinese character pronunciation and improve the mouth shape of Chinese character pronunciation, so that the transition of the mouth shape mapped to the virtual object is more natural.
[0123] Table 6 shows the situation when the finals in Pinyin are polysyllabic vowels and the initial consonants are ignored:
[0124]
[0125]
[0126] Table 6
[0127] Table 7 shows the situation when the finals in Pinyin are diphthongs and the pronunciation mouth shape of the initial consonants is ignored:
[0128]
[0129] Table 7
[0130] Table 8 shows the situation of lip-splitting of Chinese characters composed of special pinyin:
[0131]
[0132]
[0133] Table 8
[0134] Figure 3 A schematic diagram of the structure of an information mapping device provided in an embodiment of the present invention, the device mainly includes:
[0135] The pinyin splitting module 31 is used to split the pinyin of the first Chinese character into: a single first vowel, or: a first initial consonant + a second vowel, wherein the first vowel and the second vowel are: a single vowel or multiple vowels; wherein the first Chinese character is any Chinese character that the virtual object is to pronounce, and the pinyin splitting result is sent to the lip shape decomposition module 32 and the lip shape mapping module 33.
[0136] The mouth shape decomposition module 32 is used to determine the pronunciation mouth shape type of the first final or the second final obtained by the pinyin splitting module 31 according to the mouth shape change characteristics of the first final or the second final when pronouncing the first final or the second final. The pronunciation mouth shape types include: single mouth shape and compound mouth shape. If the judgment result is a compound mouth shape, the mouth shape of the first final or the second final is decomposed.
[0137] The mouth shape mapping module 33 is used to map the pronunciation mouth shape of the first vowel or the first initial consonant + the second vowel obtained by the pinyin splitting module 31 to the virtual object according to the mouth shape decomposition result obtained by the mouth shape decomposition module 32.
[0138] In an optional embodiment, the mouth shape decomposition module 32 determines that the pronunciation mouth shape type of the first vowel or the second vowel obtained by the pinyin splitting module 31 includes:
[0139] If the shape of the mouth does not change when the first vowel or the second vowel is pronounced, the pronunciation mouth shape type of the first vowel or the second vowel is determined to be a single mouth shape; otherwise, the pronunciation mouth shape type of the first vowel or the second vowel is determined to be a compound mouth shape.
[0140] In an optional embodiment, the mouth shape decomposition module 32 determines that the pronunciation mouth shape type of the first vowel or the second vowel obtained by the pinyin splitting module 31 includes:
[0141] If the first vowel or the second vowel is one of the following, the pronunciation mouth shape type of the first vowel or the second vowel is a single mouth shape: a, o, e, i, u, ü, ang, eng, ing, ong, ai, ei, ao, ou, iu, ie, üe, er, an, en, in, ün, zhi, chi, shi, ri, zi, ci, si, wu, yi, yu, ye, yue, yin, yun, ying;
[0142] If the first final or the second final is one of the following, the pronunciation mouth shape type of the first final or the second final is a compound mouth shape: ian, iao, iang, iong, uan, üan, uai, uang, ui, un, uo, ia, yuan.
[0143] In an optional embodiment, the mouth shape decomposition module 32 is further used to: classify the phonemes with the same mouth shape changes into a set according to the change rules of muscles and mouth shapes when pronouncing each initial consonant and each final, and establish a mapping relationship between the phoneme set and the mouth shape, wherein the phoneme is an initial consonant, a single final or multiple finals;
[0144] The mouth shape decomposition module 32 performs mouth shape decomposition on the first final or the second final, including: decomposing the first final or the second final into multiple phonemes according to the mapping relationship between the phoneme set and the mouth shape, and obtaining the mouth shape corresponding to each decomposed phoneme.
[0145] In an optional embodiment, the lip shape decomposition module 32 classifies the phonemes with the same lip shape changes into a set, and establishes a mapping relationship between the phoneme set and the lip shape, including:
[0146] Phoneme set 1: {a, er, an, ang, iang, ia, ai, an, ian}, corresponding to the mouth shape: open mouth with a large opening range;
[0147] Phoneme set 2: {o, ou, ong}, corresponding mouth shapes: round mouth shape;
[0148] Phoneme set three: {e}, corresponding to the mouth shape: mouth relaxed, mainly relying on the throat to pronounce;
[0149] Phoneme set 4: {d, t, n, l, g, k, h}, corresponding to the following mouth shapes: slightly open mouth, mainly relying on the tongue for pronunciation;
[0150] Phoneme set 5: {i, in, ing, y, ie, zh, ch, sh, r, z, c, s, j, q, x}, corresponding to the mouth shape: mouth slightly split backwards, mouth shape flat;
[0151] Phoneme set 6: {ei, en, eng}, the corresponding mouth shape is: open mouth, and the mouth opening amplitude is smaller than that of phoneme set 1;
[0152] Phoneme set seven: {w, u, ü, üe, un, iu, iong}, corresponding mouth shape: mouth shape gathered;
[0153] Phoneme set eight: {ao, iao}, corresponding to the mouth shape: the mouth shape is oval, and the lower lip is round;
[0154] Phoneme set nine: {b, p, m}, corresponding mouth shape: mouth basically closed.
[0155] In an optional embodiment, before the mouth shape mapping module 33 maps the pronunciation mouth shape of the first final or the first initial consonant + the second final to the virtual object, it further includes: for the first initial consonant + the second final, according to the changes in the mouth shape of the combined pronunciation of the first initial consonant and the second final, judging whether the pronunciation mouth shape of the first initial consonant is not obvious in the combined pronunciation; if so, ignoring the pronunciation mouth shape of the first initial consonant, and determining to map only the mouth shape of each phoneme decomposed from the second final to the virtual object.
[0156] In an optional embodiment, the mouth shape mapping module 33 determines whether the mouth shape of the pronunciation of the first initial consonant is not obvious in the combined pronunciation according to the mouth shape change of the combined pronunciation of the first initial consonant and the second final, including: determining whether the first initial consonant + the second final is the following combination, and if so, determining that the mouth shape of the pronunciation of the first initial consonant is not obvious in the combined pronunciation:
[0157] duan, tuan, nuan, luan, guan, kuan, huan, guai, huai, kuai, guang, kuang, huang, juan, guo, kuo, huo, luo, tuo, yuan.
[0158] In an optional embodiment, the mouth shape mapping module 33 is further used for: if the mouth shape decomposition module 32 determines that the pronunciation mouth shape type of the first final or the second final obtained by the pinyin splitting module 31 is a single mouth shape, and the first initial consonant + the second final is the following combination, then it is determined to ignore the pronunciation of the first initial consonant, and only map the pronunciation mouth shape of the second final to the virtual object: lü, wu, yu, yin, ying.
[0159] In an optional embodiment, the mouth shape decomposition module 32 performs mouth shape decomposition on the first final or the second final, including:
[0160] If the first or second final is ui, the pronunciation mouth shape of ui is decomposed into the mouth shape of u and the mouth shape of ei;
[0161] If the first or second final is un, the pronunciation mouth shape of un is decomposed into the mouth shape of u and the mouth shape of en.
[0162] In an optional embodiment, the lip shape mapping module 33 is further used for:
[0163] If the combination of the first initial consonant + the second final sent by the pinyin splitting module 31 is yun, it is determined to ignore the pronunciation of y and n, and only map the pronunciation mouth shape of u to the virtual object;
[0164] If the combination of the first initial consonant + the second final sent by the pinyin splitting module 31 is ye, it is determined to ignore the pronunciation of y, and replace the pronunciation mouth shape of e with the pronunciation mouth shape of i, and the pronunciation mouth shape of i is mapped to the virtual object as the pronunciation mouth shape of ye;
[0165] If the combination of the first initial consonant + the second final sent by the pinyin splitting module 31 is yue, it is determined to ignore the pronunciation of y and e, and only map the pronunciation mouth shape of u to the virtual object;
[0166] If the combination of the first initial consonant + the second final is zhi, chi, shi, ri, zi, ci or si, only the pronunciation mouth shape of i is mapped to the virtual object.
[0167] The embodiment of the present application also provides a computer-readable storage medium, which stores instructions, and the instructions can execute the steps in the above-mentioned information mapping method when executed by a processor. In practical applications, the computer-readable medium can be included in each device / apparatus / system in the above-mentioned embodiments, or it can exist independently without being assembled into the device / apparatus / system. Among them, the instructions are stored in the computer-readable storage medium, and the instructions stored therein can execute the steps in the above-mentioned information mapping method when executed by a processor.
[0168] According to the embodiments disclosed in the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof, but it is not used to limit the scope of protection of the present application. In the embodiments disclosed in the present application, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.
[0169] like Figure 4 As shown, an embodiment of the present invention also provides an electronic device. Figure 4 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present invention, specifically:
[0170] The electronic device may include a processor 41 with one or more processing cores, a memory 42 with one or more computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 42 is executed, the above information mapping method may be implemented.
[0171] Specifically, in practical applications, the electronic device may further include components such as a power supply 43, an input / output unit 44, etc. Those skilled in the art may understand that Figure 4 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0172] The processor 41 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the server and processes data by running or executing software programs and / or modules stored in the memory 42 and calling data stored in the memory 42, thereby monitoring the electronic device as a whole.
[0173] The memory 42 can be used to store software programs and modules, that is, the above-mentioned computer-readable storage medium. The processor 41 executes various functional applications and data processing by running the software programs and modules stored in the memory 42. The memory 42 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 42 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 42 may also include a memory controller to provide the processor 41 with access to the memory 42.
[0174] The electronic device also includes a power supply 43 for supplying power to various components, which can be logically connected to the processor 41 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 43 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, etc.
[0175] The electronic device may also include an input and output unit 44, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. The input unit output 44 may also be used to display information input by the user or information provided to the user and various graphical user interfaces, which may be composed of graphics, text, icons, videos and any combination thereof.
[0176] The flow chart and block diagram in the accompanying drawings of the present application show the possible architecture, function and operation of the system, method and computer program product according to the various embodiments disclosed in the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in the order of the standards in different figures. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0177] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, and all of these combinations and / or combinations fall within the scope disclosed in the present application.
[0178] Specific embodiments are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas, and is not used to limit the present application. For those skilled in the art, changes can be made in the specific implementation methods and application scopes according to the ideas, spirits and principles of the present invention, and any modifications, equivalent substitutions, improvements, etc. made therein should be included in the scope of protection of this application.
Claims
1. An information mapping method, characterized in that: The method includes: Splitting the pinyin of the first Chinese character into: a single first vowel, or: a first consonant + a second vowel, wherein the first vowel and the second vowel are: a single vowel or multiple vowels; wherein the first Chinese character is any Chinese character to be pronounced by the virtual object; and the virtual object is: a virtual object with a digitized appearance that can simulate pronunciation; According to the mouth shape change characteristics of the first vowel or the second vowel during pronunciation, the pronunciation mouth shape type of the first vowel or the second vowel is determined, wherein the pronunciation mouth shape type includes: a single mouth shape and a compound mouth shape; If the judgment result is a compound mouth shape, the first vowel or the second vowel is decomposed into a mouth shape; wherein, if the first vowel or the second vowel is one of the following, the pronunciation mouth shape type of the first vowel or the second vowel is a compound mouth shape: ian, iao, iang, iong, uan, üan, uai, uang, ui, un, uo, ia, yuan; According to the mouth shape decomposition result, the pronunciation mouth shape of the first vowel or the first consonant + the second vowel is mapped to the virtual object; According to the changing rules of muscles and mouth shapes when pronouncing each initial consonant and each final, phonemes with the same mouth shape changes are grouped into a set, and a mapping relationship between the phoneme set and the mouth shape is established, wherein the phonemes are initial consonants, single finals or multiple finals; The step of performing mouth shape decomposition on the first vowel or the second vowel comprises: According to the mapping relationship between the phoneme set and the mouth shape, the first final or the second final is decomposed into a plurality of phonemes, and the mouth shape corresponding to each decomposed phoneme is obtained; The step of decomposing the first vowel or the second vowel into a plurality of phonemes according to the mapping relationship between the phoneme set and the mouth shape, and obtaining the mouth shape corresponding to each of the decomposed phonemes, comprises: According to the phonemes in the mapping relationship between the phoneme set and the mouth shape, the first final or the second final is decomposed into multiple phonemes in the mapping relationship between the phoneme set and the mouth shape, and for each decomposed phoneme, its corresponding pronunciation mouth shape is searched in the mapping relationship between the phoneme set and the mouth shape.
2. The method according to claim 1, characterized in that The method of determining the pronunciation mouth shape type of the first vowel or the second vowel comprises: If the shape of the mouth does not change when the first vowel or the second vowel is pronounced, the pronunciation mouth shape type of the first vowel or the second vowel is determined to be a single mouth shape; otherwise, the pronunciation mouth shape type of the first vowel or the second vowel is determined to be a compound mouth shape.
3. The method according to claim 1 or 2, characterized in that: The method of determining the pronunciation mouth shape type of the first vowel or the second vowel comprises: If the first vowel or the second vowel is one of the following, the pronunciation mouth shape type of the first vowel or the second vowel is single mouth shape: a, o, e, i, u, ü, ang, eng, ing, ong, ai, ei, ao, ou, iu, ie, üe, er, an, en, in, ün, zhi, chi, shi, ri, zi, ci, si, wu, yi, yu, ye, yue, yin, yun, ying.
4. The method according to claim 1, characterized in that: The method of grouping the phonemes with the same mouth shape changes into a set and establishing a mapping relationship between the phoneme set and the mouth shape includes: Phoneme set 1: {a, er, an, ang, iang, ia, ai, an, ian}, corresponding to the mouth shape: open mouth with a large opening range; Phoneme set 2: {o, ou, ong}, corresponding mouth shapes: round mouth shape; Phoneme set three: {e}, corresponding to the mouth shape: mouth relaxed, mainly relying on the throat to pronounce; Phoneme set 4: {d, t, n, l, g, k, h}, corresponding to the mouth shape: slightly open mouth, mainly relying on the tongue to pronounce; Phoneme set 5: {i, in, ing, y, ie, zh, ch, sh, r, z, c, s, j, q, x}, corresponding to the mouth shape: mouth slightly split backwards, mouth shape flat; Phoneme set 6: {ei, en, eng}, the corresponding mouth shape is: open mouth, and the mouth opening amplitude is smaller than that of phoneme set 1; Phoneme set seven: {w, u, ü, üe, un, iu, iong}, corresponding mouth shape: mouth shape gathered; Phoneme set eight: {ao, iao}, corresponding to the mouth shape: the mouth shape is oval, and the lower lip is round; Phoneme set nine: {b, p, m}, corresponding mouth shape: mouth basically closed.
5. The method according to claim 1, characterized in that After obtaining the mouth shape corresponding to each decomposed phoneme and before mapping the pronunciation mouth shape of the first vowel or the first consonant + the second vowel to the virtual object, the method further includes: For the first initial consonant + the second final, determine whether the mouth shape of the first initial consonant is not obvious in the combined pronunciation based on the changes in the mouth shape of the combined pronunciation of the first initial consonant and the second final. If so, ignore the mouth shape of the first initial consonant and determine to map only the mouth shape of each phoneme decomposed from the second final to the virtual object.
6. The method according to claim 5, characterized in that The step of judging whether the mouth shape of the first initial consonant is not obvious in the combined pronunciation according to the mouth shape change of the combined pronunciation of the first initial consonant and the second final consonant comprises: Determine whether the first initial consonant + the second final is the following combination. If so, determine that the pronunciation mouth shape of the first initial consonant is not obvious in the combined pronunciation: duan, tuan, nuan, luan, guan, kuan, huan, guai, huai, kuai, guang, kuang, huang, juan, guo, kuo, huo, luo, tuo, yuan.
7. The method according to claim 1, characterized in that After determining the pronunciation mouth shape type of the first vowel or the second vowel, the method further comprises: If the judgment result is a single mouth shape, and the first initial consonant + the second final is the following combination, then the pronunciation of the first initial consonant is ignored, and only the pronunciation mouth shape of the second final is mapped to the virtual object: lü, wu, yu, yi, yin, ying.
8. The method according to claim 1, characterized in that The step of performing mouth shape decomposition on the first vowel or the second vowel comprises: If the first or second final is ui, the pronunciation mouth shape of ui is decomposed into the mouth shape of u and the mouth shape of ei; If the first or second final is un, the pronunciation mouth shape of un is decomposed into the mouth shape of u and the mouth shape of en.
9. The method according to claim 1, characterized in that: The method of splitting the pinyin of the first Chinese character into: the first initial consonant + the second final and before determining the pronunciation mouth shape type of the second final further includes: If the combination of the first initial consonant + the second final is yun, then the pronunciation of y and n is ignored, and only the pronunciation of u is mapped to the virtual object; If the combination of the first initial consonant + the second final is ye, then the pronunciation of y is ignored, and the pronunciation mouth shape of e is replaced by the pronunciation mouth shape of i, and the pronunciation mouth shape of i is mapped to the virtual object as the pronunciation mouth shape of ye; If the combination of the first initial consonant + the second final is yue, then the pronunciation of y and e is ignored, and only the pronunciation of u is mapped to the virtual object; If the combination of the first initial consonant + the second final is zhi, chi, shi, ri, zi, ci or si, only the pronunciation mouth shape of i is mapped to the virtual object.
10. An information mapping device, characterized in that: The device includes: The pinyin splitting module is used to split the pinyin of the first Chinese character into: a single first vowel, or: a first consonant + a second vowel, wherein the first vowel and the second vowel are: a single vowel or multiple vowels; wherein the first Chinese character is any Chinese character that the virtual object is going to pronounce; the virtual object is: a virtual object with a digitized appearance that can simulate pronunciation; The mouth shape decomposition module is used to judge the pronunciation mouth shape type of the first vowel or the second vowel according to the mouth shape change characteristics of the first vowel or the second vowel when pronouncing the first vowel or the second vowel, and the pronunciation mouth shape type includes: a single mouth shape and a compound mouth shape; if the judgment result is a compound mouth shape, the first vowel or the second vowel is decomposed into the mouth shape; wherein, if the first vowel or the second vowel is one of the following, the pronunciation mouth shape type of the first vowel or the second vowel is a compound mouth shape: ian, iao, iang, iong, uan, üan, uai, uang, ui, un, uo, ia, yuan; A mouth shape mapping module is used to map the pronunciation mouth shape of the first vowel or the first consonant + the second vowel to a virtual object according to the mouth shape decomposition result; The mouth shape decomposition module classifies the phonemes with the same mouth shape changes into a set according to the changing rules of muscles and mouth shapes when pronouncing each initial consonant and each final, and establishes a mapping relationship between the phoneme set and the mouth shape, wherein the phoneme is an initial consonant, a single final or multiple finals, and the mouth shape decomposition module performs mouth shape decomposition on the first final or the second final, including: decomposing the first final or the second final into multiple phonemes according to the mapping relationship between the phoneme set and the mouth shape, and obtaining the mouth shape corresponding to each decomposed phoneme; The mouth shape decomposition module decomposes the first vowel or the second vowel into a plurality of phonemes according to the mapping relationship between the phoneme set and the mouth shape, and obtains the mouth shape corresponding to each decomposed phoneme, including: According to the phonemes in the mapping relationship between the phoneme set and the mouth shape, the first final or the second final is decomposed into multiple phonemes in the mapping relationship between the phoneme set and the mouth shape, and for each decomposed phoneme, its corresponding pronunciation mouth shape is searched in the mapping relationship between the phoneme set and the mouth shape.
11. A non-transitory computer-readable storage medium storing instructions, characterized in that: When the instructions are executed by a processor, the processor is caused to perform the steps of the information mapping method according to any one of claims 1 to 9.
12. An electronic device, characterized in that: Includes the non-transitory computer-readable storage medium of claim 11, and the processor that can access the non-transitory computer-readable storage medium.
