Animation generation method, device, equipment and computer-readable storage medium

Abstract

The present application provides an animation generation method, device, equipment and computer-readable storage medium; the method includes: obtaining a virtual character animation, the virtual character animation includes audio information and text information; aligning the audio information and the text information, obtaining the syllable pronunciation start time of each Chinese character in the text information in the audio information, and obtaining at least one syllable pronunciation start time corresponding to at least one Chinese character in the text information; from the syllable lip animation, determining the Chinese character lip animation corresponding to each Chinese character, and obtaining at least one Chinese character lip animation corresponding to at least one Chinese character, the syllable lip animation is a lip animation in units of syllables; according to the pronunciation sequence corresponding to at least one Chinese character, based on at least one syllable pronunciation start time, at least one Chinese character lip animation is integrated into the virtual character animation to generate a target virtual character animation. Through the present application, the rendering effect of the animation generated in the field of artificial intelligence can be improved.

Description

Technical Field

[0001] The present application relates to computer animation technology in the field of artificial intelligence, and in particular to an animation generation method, apparatus, device and computer-readable storage medium. Background Art

[0002] With the rapid development of artificial intelligence, computer animation technology has been continuously improved, and people's requirements for lip-sync animation are getting higher and higher; therefore, when generating animation, it is necessary to match the voice in the animation with the lip shape.

[0003] Generally speaking, in order to generate an animation corresponding to speech and lip movements, a lip movement animation is usually first produced based on phonemes, and then the various lip movement animations are integrated into the virtual character animation using animation fusion technology to generate an animation corresponding to speech and lip movements. However, in the above-mentioned process of generating an animation corresponding to speech and lip movements, since Chinese characters correspond to at least one phoneme, after integrating the lip movement animations corresponding to each phoneme, the lip movement animation corresponding to each Chinese character is a combination of at least one lip movement animation corresponding to at least one phoneme, and the correspondence effect with the Chinese character is poor. Therefore, the rendering effect of the generated animation is poor. Summary of the Invention

[0004] Embodiments of the present application provide an animation generation method, apparatus, device, and computer-readable storage medium, which can improve the rendering effect of the generated animation.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present invention provides an animation generation method, including:

[0007] Acquire a virtual character animation, wherein the virtual character animation includes audio information and text information;

[0008] Aligning the audio information and the text information to obtain a syllable pronunciation start time of each Chinese character in the text information in the audio information, thereby obtaining at least one syllable pronunciation start time corresponding to at least one Chinese character in the text information;

[0009] Determining the Chinese character lip shape animation corresponding to each Chinese character from the syllable lip shape animation, thereby obtaining at least one Chinese character lip shape animation corresponding to the at least one Chinese character, wherein the syllable lip shape animation is a lip shape animation based on syllables;

[0010] According to the pronunciation sequence corresponding to the at least one Chinese character and based on the pronunciation start time of the at least one syllable, the at least one Chinese character lip animation is integrated into the virtual character animation to generate a target virtual character animation.

[0011] The present invention provides an animation generation device, including:

[0012] An animation acquisition module, used to acquire virtual character animation, wherein the virtual character animation includes audio information and text information;

[0013] a text alignment module, configured to align the audio information and the text information to obtain a syllable pronunciation start time of each Chinese character in the text information in the audio information, thereby obtaining at least one syllable pronunciation start time corresponding to at least one Chinese character in the text information;

[0014] An animation calling module is used to determine the Chinese character lip shape animation corresponding to each Chinese character from the syllable lip shape animation, thereby obtaining at least one Chinese character lip shape animation corresponding to the at least one Chinese character, wherein the syllable lip shape animation is a lip shape animation based on syllables;

[0015] The animation generation module is used to fuse the at least one Chinese character lip animation into the virtual character animation according to the pronunciation sequence corresponding to the at least one Chinese character and based on the pronunciation start time of the at least one syllable to generate a target virtual character animation.

[0016] In an embodiment of the present application, the animation generation device also includes a lip animation module, which is used to classify the initials and finals in Chinese syllables based on pronunciation similarity to obtain at least one type of initial consonant set and at least one type of final set; determine the basic initials corresponding to each type of initial consonant set in the at least one type of initial consonant set, and the basic finals corresponding to each type of final set in the at least one type of final set; combine the basic initials corresponding to each type of initial consonant set and the basic finals corresponding to each type of final set one by one to obtain the basic syllables corresponding to each type of initial consonant set and each type of final set; draw the lip animation of the basic syllable to obtain the syllable lip animation, wherein the syllable lip animation includes the lip animation of at least one basic syllable.

[0017] In an embodiment of the present application, the animation calling module is also used to split the syllables of each Chinese character to obtain a Chinese character syllable splitting result, wherein the Chinese character syllable splitting result includes at least a Chinese character syllable final; based on the Chinese character syllable splitting result, the target basic initials and target basic finals are determined from the basic initials of each type of initial consonant set in the at least one type of initial consonant set, and the basic finals of each type of final set in the at least one type of final set; the lip animation corresponding to the basic syllable composed of the target basic initials and the target basic finals is used as the Chinese character lip animation.

[0018] In an embodiment of the present application, the animation calling module is also used to, when the Chinese character syllable splitting result includes the Chinese character syllable initial consonant and the Chinese character syllable final, determine the target initial consonant set to which the Chinese character syllable initial consonant belongs from the at least one category of initial consonant set, and use the basic initial consonant corresponding to the target initial consonant set as the target basic initial consonant; determine the target final consonant set to which the Chinese character syllable final belongs from the at least one category of final consonant set, and use the basic final corresponding to the target final consonant set as the target basic final.

[0019] In an embodiment of the present application, the animation calling module is also used to, when the Chinese character syllable splitting result includes the Chinese character syllable final, determine the target initial consonant set corresponding to the Chinese character syllable final from the at least one category of initial consonant set, and use the basic initial consonants corresponding to the target initial consonant set as the target basic initial consonant; determine the target final set to which the Chinese character syllable final belongs from the at least one category of final set, and use the basic final corresponding to the target final set as the target basic final.

[0020] In an embodiment of the present application, the animation generation module is further used to determine the current Chinese character according to the pronunciation sequence corresponding to the at least one Chinese character, wherein the current Chinese character is any one of the at least one Chinese character; determine the current syllable pronunciation start time corresponding to the current Chinese character from the at least one syllable pronunciation start time; determine the current Chinese character lip animation corresponding to the current Chinese character from the at least one Chinese character lip animation; and fuse the current Chinese character lip animation at a position on the virtual character animation corresponding to the current syllable pronunciation start time until the fusion of the at least one Chinese character lip animation is completed to generate the target virtual character animation.

[0021] In an embodiment of the present application, the animation generation module is also used to obtain a syllable transition animation of a preset length when the current Chinese character is a Chinese character other than the last Chinese character in the at least one Chinese character; and splice the syllable transition animation at the end of the current Chinese character lip animation to obtain the current lip animation to be fused.

[0022] In an embodiment of the present application, the animation generation module is further configured to fuse the current lip animation to be fused at a position on the virtual character animation corresponding to the start time of pronunciation of the current syllable.

[0023] In an embodiment of the present application, the animation generating device further includes a duration acquisition module for aligning the audio information and the text information to obtain the syllable pronunciation duration of each Chinese character syllable in the audio information.

[0024] In an embodiment of the present application, the animation generation module is further used to adjust the animation speed of the Chinese character lip animation based on the syllable pronunciation duration to obtain the lip animation to be merged, thereby obtaining at least one lip animation to be merged corresponding to the at least one Chinese character lip animation; according to the pronunciation sequence corresponding to the at least one Chinese character, based on the start time of the at least one syllable pronunciation, the at least one lip animation to be merged is merged into the virtual character animation to generate the target virtual character animation.

[0025] In an embodiment of the present application, the animation generating device further includes a volume obtaining module for obtaining the syllable volume of each Chinese character in the audio information from a position on the audio information corresponding to the start time of the syllable pronunciation.

[0026] In an embodiment of the present application, the animation generation module is further used to use the syllable volume as the playback weight of the Chinese character lip animation, thereby obtaining at least one playback weight corresponding to the at least one Chinese character lip animation; according to the pronunciation sequence corresponding to the at least one Chinese character, based on the at least one syllable pronunciation start time and the at least one playback weight, the at least one Chinese character lip animation is integrated to generate the target virtual character animation.

[0027] In an embodiment of the present application, the animation generation module is further used to splice the at least one Chinese character lip animation based on the at least one syllable pronunciation start time in accordance with the pronunciation sequence corresponding to the at least one Chinese character to obtain a target lip animation; determine the earliest syllable pronunciation start time from the at least one syllable pronunciation start time; and fuse the target lip animation at a position on the virtual character animation corresponding to the earliest syllable pronunciation start time to generate the target virtual character animation.

[0028] In an embodiment of the present application, the text alignment module is further used to perform phoneme splitting on the syllables of each Chinese character in the text information to obtain phoneme information; obtain the phoneme pronunciation start time of each phoneme in the phoneme information in the audio information, thereby obtaining at least one phoneme pronunciation start time corresponding to the phoneme information; determine the target phoneme pronunciation start time corresponding to each Chinese character from the at least one phoneme pronunciation start time corresponding to the phoneme information; and determine the earliest start time of the target phoneme pronunciation start time as the syllable pronunciation start time of each Chinese character.

[0029] In an embodiment of the present application, the animation generating device further includes an animation playing module for playing the target virtual character animation in a preset virtual scene.

[0030] In an embodiment of the present application, the animation generation device also includes an animation optimization module, which is used to play the target virtual character animation in response to an animation import operation acting on the animation editing client to determine the animation to be optimized in the target virtual character animation; and optimize the animation to be optimized to obtain the optimized target virtual character animation.

[0031] In an embodiment of the present application, the animation playing module is further used to play the optimized target virtual character animation in a preset virtual scene.

[0032] An embodiment of the present application provides an animation generation device, comprising:

[0033] a memory for storing executable instructions;

[0034] The processor is used to implement the animation generation method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0035] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the animation generation method provided in the embodiment of the present application.

[0036] The embodiments of the present application have at least the following beneficial effects: when adjusting the lip shape of a virtual character in a virtual character animation, the lip shape animation is called from the syllable lip shape animation based on each Chinese character in the text information corresponding to the virtual character animation; and the syllable lip shape animation is a lip shape animation based on syllables, and since one Chinese character corresponds to one syllable; therefore, fusing the lip shape animation based on syllables can improve the correspondence between the lip shape of the generated animation and the Chinese characters, and therefore can improve the rendering effect of the generated animation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is an optional architectural diagram of the animation generation system provided in an embodiment of the present application;

[0038] Figure 2 This embodiment of the present application provides Figure 1 A schematic diagram of the composition structure of a terminal;

[0039] Figure 3 This is an optional flowchart of the animation generation method provided in the embodiment of the present application;

[0040] Figure 4 This is a schematic diagram of an exemplary playback of a target virtual character animation provided in an embodiment of the present application;

[0041] Figure 5 This is another optional flowchart of the animation generation method provided in the embodiment of the present application;

[0042] Figure 6 This is another optional flowchart of the animation generation method provided in the embodiment of the present application;

[0043] Figure 7 This is another optional flowchart of the animation generation method provided in the embodiment of the present application;

[0044] Figure 8 This is another optional flowchart of the animation generation method provided in the embodiment of the present application;

[0045] Figure 9 This is a schematic diagram of an exemplary animation generation process provided in an embodiment of the present application;

[0046] Figure 10 This is a schematic diagram of the main process of an exemplary animation generation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0048] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0050] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0051] 1) Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0052] 2) Phoneme: This is the smallest unit of speech defined by the natural properties of speech. From an acoustic perspective, a phoneme is defined by its sound quality. From a physiological perspective, a phoneme is formed by a single articulatory action. For example, "ma" consists of two articulatory actions: "m" and "a," making them two phonemes. Sounds produced by the same articulatory action are the same phoneme, while sounds produced by different articulatory actions are different phonemes. For example, in "ma-mi," the two "m"s have the same articulatory action, making them the same phoneme, while "a" and "i" have different articulatory actions, making them different phonemes. Generally speaking, phonemes are described based on articulatory actions. For example, the articulatory action for "m" involves closing the upper and lower lips, vibrating the vocal cords, and air flowing out of the nasal cavity. In phonetic terms, this is a bilabial nasal.

[0053] 3) Syllable: A phonetic unit composed of one or more phonemes, the smallest unit of phonetic structure. Syllables are composed of three parts: a head, a ventral, and a tail, resulting in distinct, perceptible boundaries between syllables. In Chinese, the pronunciation of a single character is generally considered a syllable. In Mandarin, there are 400 basic, non-tonal syllables and over 1,300 tonal syllables (excluding neutral tones). Because tone does not affect mouth shape, the syllables in the examples of this application are non-tonal syllables, but tonal syllables are also acceptable.

[0054] 4) Vertex animation: In vertex animation, each frame of animation is actually a "snapshot" of the model's specific pose. By interpolating between frames, the engine can achieve smooth animation effects.

[0055] 5) Skeletal animation. In skeletal animation, a model has a skeleton structure composed of interconnected "bones", and animation is generated for the model by changing the orientation and position of the bones. In the embodiment of the present application, the animation involved can be vertex animation or skeletal animation.

[0056] 6) Operation is a method for triggering the device to perform processing, such as a click operation, a double-click operation, a long press operation, a sliding operation, a gesture operation, a received trigger instruction, etc.; in addition, the various operations in the embodiments of the present application can be a single operation or a general term for multiple operations.

[0057] 7) In response, it is used to indicate the conditions or states on which the executed processing depends. When the dependent conditions or states are met, one or more operations executed can be real-time or have a set delay; unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0058] 8) Client: An application program running in a terminal to provide various services, such as an animation editing client; and an animation generating device is a terminal that can run a client.

[0059] It's important to note that artificial intelligence (AI) is a comprehensive technology within computer science that seeks to understand the essence of intelligence and create new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to perceive, reason, and make decisions.

[0060] Furthermore, artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0061] With the research and advancement of artificial intelligence technology, it has been studied and applied in many fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless cars, autonomous driving, drones, robots, smart healthcare, and smart customer service. As technology develops, artificial intelligence technology will be applied in more and more fields and play an increasingly important role. In the embodiments of this application, the application of artificial intelligence in the field of animation processing will be described.

[0062] Generally speaking, in order to generate animation corresponding to speech and lip shape, lip shape animation is usually first produced based on phonemes or initials and finals, and then the various lip shape animations are integrated into the virtual character animation using animation fusion technology to generate animation corresponding to speech and lip shape; however, in the above process of generating animation corresponding to speech and lip shape, since Chinese characters correspond to at least one phoneme or one initial consonant and one final, after integrating the various lip shape animations, the lip shape animation corresponding to each Chinese character is a combination of at least one lip shape animation, which has a poor correspondence effect with the Chinese characters, has an unnatural problem, and is not specifically designed according to the characteristics of Chinese; therefore, the rendering effect of the generated animation is poor.

[0063] Based on this, the embodiments of the present application provide an animation generation method, apparatus, device and computer-readable storage medium, which can improve the rendering effect of the generated animation. The following describes an exemplary application of the animation generation device provided by the embodiments of the present application. The animation generation device provided by the embodiments of the present application can be implemented as various types of user terminals such as laptop computers, tablet computers, desktop computers, set-top boxes, mobile devices (for example, mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), and can also be implemented as a server. Below, an exemplary application of the animation generation device when it is implemented as a terminal will be described.

[0064] See also Figure 1 , Figure 1 This is an optional architectural diagram of the animation generation system provided in the embodiment of the present application; Figure 1 As shown, to support an animation generation application, in an animation generation system 100, a terminal 400 (animation generation device) is connected to a server 200 via a network 300. The network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of the two. Furthermore, the animation generation system 100 also includes a database 500 for storing syllable lip animations.

[0065] Terminal 400 is used to obtain virtual character animation, wherein the virtual character animation includes audio information and text information; align the audio information and text information to obtain the syllable pronunciation start time of each Chinese character in the text information in the audio information, thereby obtaining at least one syllable pronunciation start time corresponding to at least one Chinese character in the text information; through network 300, using server 200 to call database 500 to determine the Chinese character lip animation corresponding to each Chinese character from the syllable lip animation, thereby obtaining at least one Chinese character lip animation corresponding to at least one Chinese character, wherein the syllable lip animation is a lip animation in units of syllables; according to the pronunciation sequence corresponding to at least one Chinese character, based on the at least one syllable pronunciation start time, at least one Chinese character lip animation is integrated into the virtual character animation to generate a target virtual character animation.

[0066] In some embodiments, the server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal 400 can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present invention.

[0067] See also Figure 2 , Figure 2 This embodiment of the present application provides Figure 1 A schematic diagram of the composition structure of a terminal in FIG. Figure 2 The terminal 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0068] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0069] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0070] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0071] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0072] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0073] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0074] A network communication module 452 for reaching other computing devices via one or more (wired or wireless) network interfaces 420 , exemplary network interfaces 420 including Bluetooth, Wi-Fi, and Universal Serial Bus (USB);

[0075] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripheral devices and displaying content and information);

[0076] The input processing module 454 is configured to detect one or more user inputs or interactions from one of the one or more input devices 432 and to translate the detected inputs or interactions.

[0077] In some embodiments, the animation generation device provided in the embodiments of the present application can be implemented in software. Figure 2 Animation generation device 455 stored in memory 450 is shown. This device can be software in the form of a program or plug-in, and includes the following software modules: animation acquisition module 4551, text alignment module 4552, animation call module 4553, animation generation module 4554, lip animation module 4555, duration acquisition module 4556, volume acquisition module 4557, animation playback module 4558, and animation optimization module 4559. These modules are logical and can be arbitrarily combined or further separated according to the functions implemented. The functions of each module will be described below.

[0078] In other embodiments, the animation generation device provided in the embodiments of the present application can be implemented in hardware. As an example, the animation generation device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the animation generation method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0079] Below, the animation generation method provided in the embodiment of the present application will be described in combination with the exemplary application and implementation of the terminal provided in the embodiment of the present application.

[0080] See also Figure 3 , Figure 3 This is an optional flow chart of the animation generation method provided in the embodiment of the present application, which will be combined with Figure 3 The steps shown are explained.

[0081] S301: Acquire a virtual character animation, wherein the virtual character animation includes audio information and text information.

[0082] In the embodiment of the present application, when the animation generation device performs lip-sync animation fusion, the animation generation device obtains the object to be processed, and thus obtains the virtual character animation.

[0083] It should be noted that virtual character animation is the rendered animation of a virtual character speaking in a virtual scene, including audio and text information. A virtual character is a virtual object in a virtual scene, such as a game character or a virtual target for human-computer interaction. Audio information is the voice information of the virtual character speaking, such as a voice actor dubbing the dialogue text of a game character. Text information is the text corresponding to the audio information, such as the dialogue text of a game character. Furthermore, the correspondence between the rendered lip movements of the virtual character in virtual character animation and the text and audio information is poor.

[0084] S302: Align the audio information and the text information to obtain the syllable pronunciation start time of each Chinese character in the text information in the audio information, thereby obtaining at least one syllable pronunciation start time corresponding to at least one Chinese character in the text information.

[0085] In an embodiment of the present application, after obtaining the virtual character animation, the animation generation device aligns the audio information and text information to determine the fusion position of the lip-sync animation on the virtual character animation. The time when each Chinese character in the obtained text information begins to be pronounced in the audio information, i.e., the syllable pronunciation start time, is the fusion position of the lip-sync animation on the virtual character animation. Here, the animation generation device combines the syllable pronunciation start times of all Chinese characters to obtain at least one syllable pronunciation start time corresponding to at least one Chinese character.

[0086] It should be noted that a Chinese character refers to a Chinese character in the text information, and at least one Chinese character refers to all the Chinese characters in the text information; in addition, at least one Chinese character corresponds one-to-one to at least one syllable pronunciation start time, that is, one Chinese character in at least one Chinese character corresponds to one syllable pronunciation start time in at least one syllable pronunciation start time.

[0087] S303: Determine the Chinese character lip animation corresponding to each Chinese character from the syllable lip animation, thereby obtaining at least one Chinese character lip animation corresponding to at least one Chinese character.

[0088] In an embodiment of the present application, the animation generation device is pre-configured with a syllable lip animation, or the animation generation device is able to obtain a syllable lip animation, which is a lip animation based on syllables. The animation generation device first determines the corresponding syllable for each Chinese character, and based on the determined syllable, determines the corresponding lip animation in the syllable lip animation, i.e., the Chinese character lip animation. Here, the animation generation device combines the lip animations corresponding to all Chinese characters to obtain at least one Chinese character lip animation corresponding to at least one Chinese character.

[0089] It should be noted that the syllable lip animation can be the lip animation corresponding to each Chinese character syllable, wherein each Chinese character syllable can be a toned syllable or an untonic syllable; the syllable lip animation can also be the lip animation corresponding to the basic syllable, wherein the basic syllable is a representative syllable of one or more Chinese character syllables; the embodiments of the present application do not make specific limitations on this.

[0090] S304: According to the pronunciation sequence corresponding to the at least one Chinese character, based on the pronunciation start time of at least one syllable, at least one Chinese character lip animation is integrated into the virtual character animation to generate a target virtual character animation.

[0091] In an embodiment of the present application, the animation generation device can determine the pronunciation order of at least one Chinese character based on audio information, or can determine the pronunciation order of at least one syllable corresponding to at least one Chinese character, and the pronunciation order is the pronunciation sequence. The animation generation device determines the fusion position based on the pronunciation start time of at least one syllable in accordance with the pronunciation sequence, and fuses at least one Chinese character lip animation into the virtual character animation. After the fusion is completed, the target virtual character animation is generated. Here, the animation generation device can first fuse at least one Chinese character lip animation, and then fuse the fused at least one Chinese character lip animation into the virtual character animation to generate the target virtual character; it can also fuse at least one Chinese character lip animation into the virtual character animation one by one to generate the target virtual character animation; the embodiment of the present application does not make specific limitations on this.

[0092] It should be noted that the rendered lip shape of the virtual character in the target virtual character animation has good correspondence with the text information and the audio information.

[0093] For example, see Figure 4 , Figure 4 : is a schematic diagram of an exemplary playback of a target virtual character animation provided in an embodiment of the present application; Figure 4 As shown, the rendered lip shapes 4-1 to 4-12 are obtained by taking screenshots of the target virtual character animation being played.

[0094] It can be understood that since the at least one Chinese character lip-sync animation obtained is realized by obtaining the Chinese character lip-sync animation corresponding to each Chinese character from the syllable lip-sync animation; and the syllable lip-sync animation is a lip-sync animation based on syllables, one Chinese character corresponds to one syllable, so the correspondence between each Chinese character lip-sync animation and the Chinese character is better, which can also improve the rendering effect of the target virtual character animation.

[0095] See also Figure 5 , Figure 5 This is another optional flow chart of the animation generation method provided in the embodiment of the present application; Figure 5 As shown, in the embodiment of the present application, S305-S308 are also included before S303; that is, before the animation generation device determines the Chinese character lip animation corresponding to each Chinese character from the syllable lip animation, the animation generation method also includes S305-S308, and each step is explained below.

[0096] S305: Based on pronunciation similarity, the initials and finals in the Chinese syllables are classified respectively to obtain at least one type of initial consonant set and at least one type of final set.

[0097] In an embodiment of the present application, the animation generation device classifies the initial consonants and finals in Chinese syllables based on pronunciation similarity, so that for the initial consonants in the Chinese syllables, at least one type of initial consonant set can be obtained; for the finals in the Chinese syllables, at least one type of final set can be obtained.

[0098] It should be noted that pronunciation similarity can refer to similarity in the pronunciation mouth shape or similarity in the phonetic characteristics of the pronunciation, and this is not specifically limited in the present embodiment. Furthermore, the number of categories corresponding to at least one set of initial consonants and at least one set of finals can be determined based on actual circumstances. However, the greater the number of categories corresponding to at least one set of initial consonants and at least one set of finals, the more lip animations included in the determined syllable lip animation.

[0099] S306: Determine the basic initials corresponding to each type of initial consonant set in at least one type of initial consonant set, and the basic finals corresponding to each type of final set in at least one type of final set.

[0100] It should be noted that, for each set of initial consonants in at least one set of initial consonants, the animation generating device determines an initial consonant from the set of initial consonants as the basic initial consonant in the set of initial consonants; for example, for the set of initial consonants "b, p, m", the corresponding basic initial consonant is "b". In addition, for each set of finals in at least one set of finals, the animation generating device determines a final from the set of finals as the basic final in the set of finals; for example, for the set of finals "a, ia, an, ang, ai, ian, iang, ua, uan, uang", the corresponding basic final is "a".

[0101] S307 , combining the basic initial consonants corresponding to each type of initial consonant set and the basic final consonants corresponding to each type of final consonant set one by one to obtain basic syllables corresponding to each type of initial consonant set and each type of final consonant set.

[0102] It should be noted that the number of basic syllables obtained by the animation generation device can be the product of the number of categories of at least one type of initial consonant set and the number of categories of at least one type of final vowel set, or it can be less than the product of the number of categories of at least one type of initial consonant set and the number of categories of at least one type of final vowel set (for example, some basic initial consonants and some basic finals cannot form syllables). The embodiments of the present application do not make specific limitations on this.

[0103] S308: Draw the lip animation of the basic syllable to obtain the syllable lip animation.

[0104] It should be noted that the syllable lip animation includes the lip animation of at least one basic syllable, wherein the number corresponding to at least one basic syllable is less than or equal to the product of the number of categories of at least one type of initial consonant set and the number of categories of at least one type of final set.

[0105] Accordingly, in the embodiment of the present application, the animation generation device in S303 determines the Chinese character lip animation corresponding to each Chinese character from the syllable lip animation, which can be achieved through S3031-S3033. Each step is explained below.

[0106] S3031. Split the syllables of each Chinese character to obtain a Chinese character syllable splitting result.

[0107] It should be noted that the animation generation device splits each Chinese character syllable based on the syllable's initial consonant and final vowel, and the resulting split result is the Chinese character syllable split result, wherein the Chinese character syllable split result at least includes the Chinese character syllable final vowel.

[0108] S3032. Based on the result of Chinese character syllable splitting, determine target basic initials and target basic finals from the basic initials of each category of initial consonant sets in at least one category of initial consonant sets, and the basic finals of each category of final sets in at least one category of final sets.

[0109] It should be noted that the target basic initials are the basic initials corresponding to the result of Chinese character syllable splitting among the basic initials; the target basic finals are the basic finals corresponding to the result of Chinese character syllable splitting among the basic finals.

[0110] S3033: Use the lip animation corresponding to the basic syllable composed of the target basic initial consonant and the target basic final as the Chinese character lip animation.

[0111] It should be noted that the lip animation corresponds to the basic syllable. The target basic initial consonant and the target basic final can constitute the basic syllable. Therefore, the animation generation device can obtain the lip animation corresponding to the corresponding basic syllable based on the target basic initial consonant and the target basic final, that is, the Chinese character lip animation corresponding to each Chinese character.

[0112] It can be understood that the animation generation device classifies the initials and finals in Chinese syllables separately to obtain syllable lip animations including lip animations corresponding to basic syllables, so that the number of lip animations included in the syllable lip animation is smaller, thereby reducing the resource consumption of obtaining the lip animation corresponding to the syllable lip animation and improving the efficiency of obtaining the syllable lip animation.

[0113] In an embodiment of the present application, S3032 can be implemented through S30321 and S30322; that is, the animation generation device determines the target basic initials and target basic finals from the basic initials of each category of initial consonant sets in at least one category of initial consonant sets, and the basic finals of each category of final sets in at least one category of final sets based on the results of Chinese character syllable splitting, including S30321 and S30322. Each step is explained below.

[0114] S30321. When the result of splitting a Chinese character syllable includes a Chinese character syllable initial consonant and a Chinese character syllable final consonant, determine the target initial consonant set to which the Chinese character syllable initial consonant belongs from at least one type of initial consonant set, and use the basic initial consonant corresponding to the target initial consonant set as the target basic initial consonant.

[0115] It should be noted that, since the result of splitting a Chinese character syllable includes at least the final vowel of a Chinese character syllable, there are cases where the result of splitting a Chinese character syllable includes the final vowel of a Chinese character syllable and the initial consonant of a Chinese character syllable, and there are also cases where the result of splitting a Chinese character syllable includes the final vowel of a Chinese character syllable. In the case where the result of splitting a Chinese character syllable includes the initial consonant of a Chinese character syllable and the final vowel of a Chinese character syllable, the animation generation device first determines a class of initial consonant sets to which the Chinese character syllable initial consonant belongs in at least one class of initial consonant sets, namely, the target initial consonant set; since each class of initial consonant sets corresponds to a basic initial consonant, the target initial consonant set also corresponds to a basic initial consonant, and here, the basic initial consonant corresponding to the target initial consonant set is the target basic initial consonant; at this point, the animation generation device has completed the determination of the target basic initial consonant corresponding to the result of splitting a Chinese character syllable.

[0116] S30322. Determine a target final set to which the Chinese character syllable final belongs from at least one type of final set, and use the basic final corresponding to the target final set as the target basic final.

[0117] It should be noted that the animation generation device first determines a category of finals to which the Chinese character syllable initial consonant belongs in at least one category of finals set, namely, the target finals set; since each category of finals set corresponds to a basic final, the target final set also corresponds to a basic final. Here, the basic final corresponding to the target final set is the target basic final; at this point, the animation generation device has completed the determination of the target basic final corresponding to the result of the Chinese character syllable splitting.

[0118] In an embodiment of the present application, S3032 can also be implemented through S30323 and S30324; that is, the animation generation device determines the target basic initials and target basic finals from the basic initials of each category of initial consonant sets in at least one category of initial consonant sets, and the basic finals of each category of final sets in at least one category of final sets based on the results of Chinese character syllable splitting, including S30323 and S30324. Each step is explained below.

[0119] S30323. When the Chinese character syllable splitting result includes the Chinese character syllable's final, determine the target initial set corresponding to the Chinese character syllable's final from at least one type of initial set, and use the basic initial corresponding to the target initial set as the target basic initial.

[0120] It should be noted that there is one type of initial set in at least one type of initial set, which includes not only each initial but also syllables without an initial. Such syllables without an initial are, for example, "o, en, a". For the case where the Chinese character syllable splitting result includes the Chinese character syllable's final, since the syllable of this Chinese character in the text information at this time is a single-final syllable; here, the animation generation device determines the corresponding target basic initial from at least one basic initial corresponding to at least one type of initial set based on the Chinese character syllable's final; that is to say, the animation generation device first determines the type of initial set to which the Chinese character syllable's final belongs in at least one type of initial set, that is, the target initial set; since each type of initial set corresponds to a basic initial, the target initial set also corresponds to a basic initial. Here, the basic initial corresponding to the target initial set is the target basic initial; at this time, the animation generation device has completed the determination of the target basic initial corresponding to the Chinese character syllable splitting result.

[0121] Exemplarily, this type of initial set that includes not only each initial but also syllables without an initial is the set of other initials except "b, p, m, f" and syllables without an initial, and the corresponding basic initial is "d". Thus, for the Chinese character "啊" in the text information, the corresponding target basic initial is "d".

[0122] S30324. Determine the target final set to which the Chinese character syllable's final belongs from at least one type of final set, and use the basic final corresponding to the target final set as the target basic final.

[0123] It should be noted that the description of the implementation process of S30324 is the same as the description of the implementation process of S30322, and the embodiments of this application will not repeat it here.

[0124] It can be understood that whether it is a Chinese character syllable including an initial and a final or a Chinese character syllable only including a final, the corresponding target basic initial and target basic final can be determined. Furthermore, the basic syllable can be determined through the target basic initial and target basic final to obtain the mouth shape animation corresponding to the basic syllable, reducing the resource consumption for obtaining the syllable mouth shape animation and improving the acquisition efficiency of the Chinese character mouth shape animation.

[0125] In an embodiment of the present application, S304 can be implemented through S3041-S3044; that is, the animation generation device integrates at least one Chinese character lip animation into the virtual character animation based on the pronunciation start time of at least one syllable in accordance with the pronunciation sequence corresponding to at least one Chinese character, and generates the target virtual character animation, including S3041-S3044. Each step is explained below.

[0126] S3041. Determine the current Chinese character according to the pronunciation sequence corresponding to at least one Chinese character.

[0127] It should be noted that the animation generating device traverses at least one Chinese character according to the pronunciation sequence corresponding to the at least one Chinese character, and the traversed Chinese character is the current Chinese character. Therefore, the current Chinese character is any one of the at least one Chinese character.

[0128] S3042. Determine the pronunciation start time of the current syllable corresponding to the current Chinese character from at least one syllable pronunciation start time.

[0129] In an embodiment of the present application, since each Chinese character corresponds to a syllable pronunciation start time, the animation generation device can determine the syllable pronunciation start time corresponding to the current Chinese character from at least one syllable pronunciation start time, thereby determining the current syllable pronunciation start time corresponding to the current Chinese character.

[0130] S3043: Determine the current Chinese character lip animation corresponding to the current Chinese character from at least one Chinese character lip animation.

[0131] In the embodiment of the present application, since each Chinese character corresponds to a Chinese character lip animation, the animation generation device can determine the Chinese character lip animation corresponding to the current Chinese character from at least one Chinese character lip animation, and thus determine the current Chinese character lip animation corresponding to the current Chinese character.

[0132] S3044. At a position on the virtual character animation corresponding to the start time of the current syllable pronunciation, fuse the current Chinese character lip animation until the fusion of at least one Chinese character lip animation is completed to generate the target virtual character animation.

[0133] It should be noted that, since the starting time of the current syllable pronunciation is the starting time of the current Chinese character pronunciation in the audio information of the virtual character animation, the animation generation device can determine the position corresponding to the starting time of the current syllable pronunciation on the virtual character animation; thus, the animation generation device merges the current Chinese character lip animation into the virtual character animation at the determined position; and the lip animation of each Chinese character is merged one by one until the fusion of at least one Chinese character lip animation is completed, and the fused virtual character animation is the target virtual character animation.

[0134] In an embodiment of the present application, after the animation generation device determines the current Chinese character mouth shape animation corresponding to the current Chinese character from at least one Chinese character mouth shape animation in S3043, the animation generation method further includes S3045 and S3046, and each step will be described separately below.

[0135] S3045. When the current Chinese character is a Chinese character other than the last Chinese character among at least one Chinese character, obtain a syllable transition animation with a preset duration.

[0136] In an embodiment of the present application, the animation generation device also adds a syllable transition animation with a preset duration between two Chinese character mouth shape animations; thus, when the current Chinese character is a Chinese character other than the last Chinese character among at least one Chinese character, the animation generation device obtains the syllable transition animation with a preset duration.

[0137] It should be noted that the preset duration is less than the duration of the Chinese character mouth shape animation. For example, 0.1 second or 0.2 second; the syllable transition animation is obtained based on the current Chinese character mouth shape animation and the Chinese character mouth shape animation of the next Chinese character, and is a transition animation from the current Chinese character mouth shape animation to the Chinese character mouth shape animation of the next Chinese character; for example, when the current Chinese character is "you" and the next Chinese character is "hao", the syllable transition animation is the common sound of "you" and "hao".

[0138] S3046. Splices the syllable transition animation at the end of the current Chinese character mouth shape animation to obtain the current mouth shape animation to be fused.

[0139] It should be noted that the animation generation device can splice the syllable transition animation at the end of the current Chinese character mouth shape animation, or splice the syllable transition animation at the beginning of the Chinese character mouth shape animation of the next Chinese character. The embodiment of the present application does not make specific limitations on this.

[0140] In addition, the animation generation device can determine the position of the preset duration in the virtual character animation by shortening the duration of the current Chinese character mouth shape animation and / or the duration of the Chinese character mouth shape animation of the next Chinese character, so as to fuse the syllable transition animation.

[0141] Correspondingly, in an embodiment of the present application, in S3044, when the animation generation device fuses the current Chinese character mouth shape animation at the position corresponding to the start time of the current syllable pronunciation in the virtual character animation, it includes: the animation generation device fuses the current mouth shape animation to be fused at the position corresponding to the start time of the current syllable pronunciation in the virtual character animation. That is to say, when the animation generation device splices the syllable transition animation to the current Chinese character mouth shape animation and obtains the current mouth shape animation to be fused, the current mouth shape animation to be fused is fused into the virtual character animation.

[0142] It is understandable that by adding a syllable transition animation between the two Chinese character lip animations, the fused lip animation becomes more natural, which solves the jitter problem caused by the instantaneous switching between the two Chinese character lip animations and improves the rendering effect of the fused lip animation.

[0143] See also Figure 6 , Figure 6 This is another optional flow chart of the animation generation method provided in the embodiment of the present application; Figure 6 As shown, in the embodiment of the present application, S309 is further included after S301; that is, after the animation generation device obtains the virtual character animation, the animation generation method further includes S309, and this step is described below.

[0144] S309: Align the audio information and the text information to obtain the pronunciation duration of each Chinese character syllable in the audio information.

[0145] In an embodiment of the present application, the syllables of each Chinese character in the text information are decomposed into phonemes to obtain phoneme information; the phoneme pronunciation duration of each phoneme in the phoneme information in the audio information is obtained, thereby obtaining at least one phoneme pronunciation duration corresponding to the phoneme information; from the at least one phoneme pronunciation duration corresponding to the phoneme information, the target phoneme pronunciation duration corresponding to each Chinese character is determined; and the cumulative duration of the target phoneme pronunciation duration is determined as the syllable pronunciation duration of each Chinese character. Here, the phoneme information is a collection of individual phonemes corresponding to at least one Chinese character, and the target phoneme pronunciation duration includes at least one phoneme pronunciation duration.

[0146] It should be noted that the alignment of audio information and text information in S302 and S309 is the same processing process. Therefore, the animation generation device aligns the audio information and text information to obtain the syllable pronunciation duration and syllable pronunciation start time of each Chinese syllable in the audio information.

[0147] Continue to see Figure 6 Accordingly, in the embodiment of the present application, S304 can also be implemented through S3047 and S3048; that is, the animation generation device integrates at least one Chinese character lip animation into the virtual character animation based on the pronunciation start time of at least one syllable in accordance with the pronunciation sequence corresponding to at least one Chinese character, and generates the target virtual character animation, including S3047 and S3048. Each step is explained below.

[0148] S3047: Adjust the animation speed of the Chinese character lip animation based on the syllable pronunciation duration to obtain the lip animation to be merged, thereby obtaining at least one lip animation to be merged corresponding to the at least one Chinese character lip animation.

[0149] In the embodiment of the present application, since the duration of the Chinese character lip animation is fixed, it is not necessarily equal to the syllable pronunciation duration; therefore, the animation generation device adjusts the animation speed of the Chinese character lip animation based on the syllable pronunciation duration, so that the duration of the adjusted Chinese character lip animation is the syllable pronunciation duration, and the adjusted Chinese character lip animation is the lip animation to be fused; thus, when the animation generation device completes the animation speed adjustment of at least one Chinese character lip animation, at least one lip animation to be fused is obtained.

[0150] It should be noted that at least one Chinese character lip-sync animation corresponds to at least one lip-sync animation to be merged.

[0151] S3048. According to the pronunciation sequence corresponding to at least one Chinese character, based on the pronunciation start time of at least one syllable, at least one lip animation to be merged is merged into the virtual character animation to generate a target virtual character animation.

[0152] It should be noted that after the animation generating device completes the adjustment of the animation speed of at least one Chinese character lip-sync animation and obtains at least one lip-sync animation to be merged, the at least one lip-sync animation to be merged will be merged into the virtual character animation.

[0153] See also Figure 7 , Figure 7 This is another optional flow chart of the animation generation method provided in the embodiment of the present application; Figure 7 As shown, in the embodiment of the present application, S302 also includes S310; that is, the animation generation device aligns the audio information and the text information to obtain the pronunciation start time of the syllable of each Chinese character in the text information after the audio information. The animation generation method also includes S310, and this step is explained below.

[0154] S310. Obtain the syllable volume of each Chinese character in the audio information from the position on the audio information corresponding to the start time of the syllable pronunciation.

[0155] It should be noted that the syllable volume can be the maximum pronunciation volume of each Chinese character in the audio information, or it can be the volume record of the pronunciation duration corresponding to each Chinese character, etc. The embodiment of the present application does not make specific limitations on this.

[0156] Continue to see Figure 7 Accordingly, in the embodiment of the present application, S304 can also be implemented through S3049 and S30410; that is, the animation generation device integrates at least one Chinese character lip animation into the virtual character animation based on the pronunciation start time of at least one syllable in accordance with the pronunciation sequence corresponding to at least one Chinese character, and generates the target virtual character animation, including S3049 and S30410. Each step is explained below.

[0157] S3049: Using the syllable volume as the playback weight of the Chinese character lip-sync animation, thereby obtaining at least one playback weight corresponding to at least one Chinese character lip-sync animation.

[0158] It should be noted that the playback weight is a playback parameter of the Chinese character lip-sync animation, which is used to measure the playback degree of the Chinese character lip-sync animation. For example, when the playback weight is greater than 1, the lip shape of the Chinese character lip-sync animation is larger, and when the playback weight is less than 1, the lip shape of the Chinese character lip-sync animation is smaller. In addition, at least one Chinese character lip-sync animation is in a one-to-one correspondence with at least one playback weight.

[0159] S30410. According to the pronunciation sequence corresponding to at least one Chinese character, based on at least one syllable pronunciation start time and at least one playback weight, at least one Chinese character lip animation is integrated to generate a target virtual character animation.

[0160] It should be noted that after the animation generation device obtains at least one playback weight corresponding to at least one Chinese character lip animation, it integrates at least one Chinese character lip animation into the virtual character animation based on at least one syllable pronunciation start time and at least one playback weight.

[0161] It can be understood that by using the syllable volume of each Chinese character as the playback weight of the Chinese character lip animation, different Chinese characters correspond to different playback weights, so that the rendering effect of the target virtual object animation obtained is better.

[0162] In an embodiment of the present application, the animation generation device can integrate Chinese character lip-sync animations into the virtual character animation one by one, as described in S3041-S3044. The animation generation device can also first splice at least one Chinese character lip-sync animation, and then integrate the spliced ​​at least one Chinese character lip-sync animation into the virtual character animation to obtain the target virtual character animation. Therefore, S304 can also be implemented through S30411-S30413; that is, the animation generation device integrates at least one Chinese character lip-sync animation into the virtual character animation based on the pronunciation start time of at least one syllable according to the pronunciation sequence corresponding to at least one Chinese character, to generate the target virtual character animation, including S30411-S30413. Each step is described below.

[0163] S30411. According to the pronunciation sequence corresponding to at least one Chinese character, based on the pronunciation start time of at least one syllable, splice at least one Chinese character lip animation to obtain a target lip animation.

[0164] It should be noted that when the animation generation device splices at least one Chinese character lip-sync animation according to the pronunciation sequence, it can also determine at least one playback weight corresponding to the at least one Chinese character lip-sync animation based on the syllable volume of each Chinese character, and can also adjust the animation speed of the at least one Chinese character lip-sync animation based on the syllable pronunciation duration of each Chinese character, and can also integrate syllable transition animations between any two Chinese character lip-sync animations in the at least one Chinese character lip-sync animation.

[0165] S30412. Determine the earliest syllable pronunciation start time from at least one syllable pronunciation start time.

[0166] In the embodiment of the present application, the animation generating device selects the earliest syllable pronunciation starting time from at least one syllable pronunciation starting time, thereby obtaining the earliest syllable pronunciation starting time.

[0167] S30413. At a position on the virtual character animation corresponding to the start time of the earliest syllable pronunciation, the target lip animation is integrated to generate the target virtual character animation.

[0168] In an embodiment of the present application, after the animation generation device completes the splicing of at least one lip animation to obtain the target lip animation, it merges the target lip animation into the position on the virtual character animation corresponding to the start time of the earliest syllable pronunciation, thereby generating the target virtual character animation.

[0169] In an embodiment of the present application, in S302, the animation generation device aligns the audio information and the text information to obtain the syllable pronunciation start time of each Chinese character in the text information in the audio information, including: the animation generation device splits the syllable of each Chinese character in the text information into phonemes to obtain phoneme information; and obtains the phoneme pronunciation start time of each phoneme in the phoneme information in the audio information, thereby obtaining at least one phoneme pronunciation start time corresponding to the phoneme information; and determines the target phoneme pronunciation start time corresponding to each Chinese character from the at least one phoneme pronunciation start time corresponding to the phoneme information; and determines the earliest start time of the target phoneme pronunciation start time as the syllable pronunciation start time of each Chinese character.

[0170] For example, the animation generating device may perform alignment through MFA (Montreal-Forced-Aligner) or “LibriSpeech” or “speech-aligner”.

[0171] See also Figure 8 , Figure 8 This is another optional flow chart of the animation generation method provided in the embodiment of the present application; Figure 8As shown, in the embodiment of the present application, S304 also includes S311; that is, after the animation generation device generates the target virtual character animation, the animation generation method also includes S311, which is described below.

[0172] S311: Play the target virtual character animation in the preset virtual scene.

[0173] It should be noted that the animation generating device plays the target virtual character animation in the rendered preset virtual scene to complete the rendering of the lip shape of the virtual character animation.

[0174] Continue to see Figure 8 Accordingly, in the embodiment of the present application, S312 and S313 are also included before S311; that is, before the animation generation device plays the target virtual character animation in the preset virtual scene, the animation generation method also includes S312 and S313. Each step is explained below.

[0175] S312: In response to the animation import operation performed on the animation editing client, the target virtual character animation is played to determine the animation to be optimized in the target virtual character animation.

[0176] In an embodiment of the present application, when a user imports a target virtual character animation into an animation editing client, the animation generation device also receives the animation import operation performed on the animation editing client. At this time, the animation generation device responds to the animation import operation by importing the target virtual character animation into the animation editing client and playing the target virtual character animation. Here, the animation generation device plays the target virtual character animation to determine the animation to be optimized in the target virtual character animation by receiving the operation.

[0177] S313: Optimize the animation to be optimized to obtain an optimized target virtual character animation.

[0178] It should be noted that the animation generation device can adjust the lip shape of the animation to be optimized in the target virtual character animation, and can also add expressions to the animation to be optimized in the target virtual character animation. The embodiment of the present application does not specifically limit the optimization method.

[0179] Continue to see Figure 8 Accordingly, in the embodiment of the present application, S311 can be implemented through S3111; that is, the animation generation device plays the target virtual character animation in the preset virtual scene, including S3111, and this step is explained below.

[0180] S3111. Play the optimized target virtual character animation in the preset virtual scene.

[0181] That is, after the animation generating device completes the optimization of the animation to be optimized in the target virtual character animation and obtains the optimized target virtual character animation, the optimized target virtual character animation is played in the preset virtual scene.

[0182] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0183] See also Figure 9 , Figure 9 This is a schematic diagram of an exemplary animation generation process provided by an embodiment of the present application; Figure 9 As shown, when the scene corresponding to the virtual character dialogue animation (virtual character animation) is a scene in which a virtual character in a virtual scene performs dialogue, the exemplary animation generation process includes the following steps:

[0184] S901, start.

[0185] S902: Acquire the dialogue audio (audio information) and dialogue text (text information) of the virtual character's dialogue animation.

[0186] S903: Align the dialogue audio and the dialogue text to obtain the pronunciation start time (syllable pronunciation start time) and pronunciation duration (syllable pronunciation duration) of each Chinese character in the dialogue text in the dialogue audio.

[0187] S904: Based on the syllables of each Chinese character, call the basic syllable animation (syllable lip animation) to obtain the lip animation (Chinese character lip animation) corresponding to each Chinese character.

[0188] It should be noted that the production process of basic syllable animation includes: classification of initials and finals, classification of Chinese characters, and production of basic syllable animation.

[0189] Among them, when classifying initials and finals, the initials and finals are classified separately according to their pronunciation similarity, and the resulting initial consonant categories are shown in Table 1:

[0190] Table 1

[0191] Initial consonant representative Initials that can be represented b b, p, m f f d All other initials and syllables without initials

[0192] It should be noted that the initial consonants represented are the basic initial consonants in the embodiment of the present application, and the initial consonants that can be represented are the initial consonant sets in the embodiment of the present application.

[0193] The resulting final categories are shown in Table 2:

[0194] Table 2

[0195] Finals The finals that can be represented a a, ia, an, ang, ai, ian, iang, ua, uan, uang e e, o, er, ie, ei, en, eng i i, in, ing u u, ü, iu ao ao,ou,iao,ong uo uo, üe, uai, ui, üan, un, ün

[0196] It should be noted that the finals represented are the basic finals in the embodiment of the present application, and the finals that can be represented are the final set in the embodiment of the present application.

[0197] When classifying Chinese characters, we classify them into 14 representative Chinese characters based on the initial consonant representatives and the final vowel representatives (the corresponding syllables are the basic syllables), as shown in Table 3:

[0198] Table 3

[0199]

[0200]

[0201] When making basic syllable animations, corresponding lip animations are made for the above 14 Chinese characters to obtain basic syllable animations.

[0202] S905. Adjust the pronunciation duration based on the duration of the transition animation (syllable transition animation) between the lip animations corresponding to the two Chinese characters to obtain an adjusted duration.

[0203] For example, the duration of pronunciation minus half of the duration of the transition animation is used as the adjustment duration; that is, when the duration of the transition animation is 0.2 seconds, the duration of "you" in the dialogue text "hello" is 0.3 seconds and the duration of "good" is 0.5 seconds, then the adjusted duration of "you" is 0.2 seconds and the adjusted duration of "you" is 0.4 seconds.

[0204] S906: Adjust the animation speed of the lip animation corresponding to each Chinese character based on the adjustment duration.

[0205] S907: Based on the pronunciation start time, obtain the volume of each Chinese character from the dialogue audio.

[0206] S908: merging a transition animation between the two adjusted lip animations. In other words, merging a transition animation between the lip animations corresponding to the two Chinese characters.

[0207] S909: Using the volume of each Chinese character as the playback weight of the fused lip animation (the lip animation after fusion of transition animation and adjustment of animation speed), and fusion into the virtual character dialogue animation to obtain the final virtual character dialogue animation.

[0208] S910, end.

[0209] Based on S901-S910, the main process of exemplary animation generation can be determined, such as Figure 10As shown, first, for the virtual character dialogue animation 10-1, the dialogue audio and dialogue text 10-2 are obtained; secondly, the dialogue audio and dialogue text 10-2 are aligned to obtain the pronunciation start time and pronunciation duration 10-3 of each Chinese character; then, for each Chinese character, the lip animation 10-5 is called from the basic syllable animation 10-4; finally, the lip animation 10-5 is integrated into the virtual character dialogue animation 10-1 using the start time and duration 10-3 of the syllable of each Chinese character to obtain the final virtual character dialogue animation 10-6.

[0210] It is understood that the animation generation method provided in the embodiments of the present application solves the problem of unnatural Chinese pronunciation caused by phoneme-based animation generation methods, and improves the overall performance of the lip movements in the generated animation. In other words, by taking advantage of the characteristic of Chinese pronunciation that each character corresponds to a syllable and integrating lip movement animation on a syllable basis, the Chinese lip movement animation can be made to exhibit a more natural visual effect, improving the rendering effect of the animation.

[0211] The following continues to describe the exemplary structure of the animation generating device 455 provided in the embodiment of the present application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the animation generating device 455 of the memory 450 may include:

[0212] Animation acquisition module 4551, used to acquire virtual character animation, wherein the virtual character animation includes audio information and text information;

[0213] a text alignment module 4552 configured to align the audio information and the text information to obtain a syllable pronunciation start time of each Chinese character in the text information in the audio information, thereby obtaining at least one syllable pronunciation start time corresponding to at least one Chinese character in the text information;

[0214] Animation calling module 4553 is used to determine the Chinese character lip animation corresponding to each Chinese character from the syllable lip animation, thereby obtaining at least one Chinese character lip animation corresponding to the at least one Chinese character, wherein the syllable lip animation is a lip animation based on syllables;

[0215] The animation generation module 4554 is used to integrate the at least one Chinese character lip animation into the virtual character animation based on the pronunciation start time of the at least one syllable according to the pronunciation sequence corresponding to the at least one Chinese character to generate the target virtual character animation.

[0216] In an embodiment of the present application, the animation generating device 455 also includes a lip animation module 4555, which is used to classify the initials and finals in Chinese syllables based on pronunciation similarity, and obtain at least one type of initial consonant set and at least one type of final set; determine the basic initials corresponding to each type of initial consonant set in the at least one type of initial consonant set, and the basic finals corresponding to each type of final set in the at least one type of final set; combine the basic initials corresponding to each type of initial consonant set and the basic finals corresponding to each type of final set one by one to obtain the basic syllables corresponding to each type of initial consonant set and each type of final set; draw the lip animation of the basic syllable to obtain the syllable lip animation, wherein the syllable lip animation includes the lip animation of at least one basic syllable.

[0217] In an embodiment of the present application, the animation calling module 4553 is also used to split the syllables of each Chinese character to obtain a Chinese character syllable splitting result, wherein the Chinese character syllable splitting result includes at least a Chinese character syllable final; based on the Chinese character syllable splitting result, the target basic initials and target basic finals are determined from the basic initials of each type of initial consonant set in the at least one type of initial consonant set, and the basic finals of each type of final set in the at least one type of final set; the lip animation corresponding to the basic syllable composed of the target basic initials and the target basic finals is used as the Chinese character lip animation.

[0218] In an embodiment of the present application, the animation calling module 4553 is also used to, when the Chinese character syllable splitting result includes the Chinese character syllable initial consonant and the Chinese character syllable final, determine the target initial consonant set to which the Chinese character syllable initial consonant belongs from the at least one category of initial consonant set, and use the basic initial consonant corresponding to the target initial consonant set as the target basic initial consonant; determine the target final consonant set to which the Chinese character syllable final belongs from the at least one category of final consonant set, and use the basic final corresponding to the target final consonant set as the target basic final.

[0219] In an embodiment of the present application, the animation calling module 4553 is also used to, when the Chinese character syllable splitting result includes the Chinese character syllable final, determine the target initial consonant set corresponding to the Chinese character syllable final from the at least one category of initial consonant set, and use the basic initial consonants corresponding to the target initial consonant set as the target basic initial consonant; determine the target final set to which the Chinese character syllable final belongs from the at least one category of final set, and use the basic final corresponding to the target final set as the target basic final.

[0220] In an embodiment of the present application, the animation generation module 4554 is further used to determine the current Chinese character according to the pronunciation sequence corresponding to the at least one Chinese character, wherein the current Chinese character is any one of the at least one Chinese character; determine the current syllable pronunciation start time corresponding to the current Chinese character from the at least one syllable pronunciation start time; determine the current Chinese character lip animation corresponding to the current Chinese character from the at least one Chinese character lip animation; and fuse the current Chinese character lip animation at a position on the virtual character animation corresponding to the current syllable pronunciation start time until the fusion of the at least one Chinese character lip animation is completed to generate the target virtual character animation.

[0221] In an embodiment of the present application, the animation generation module 4554 is also used to obtain a syllable transition animation of a preset length when the current Chinese character is a Chinese character other than the last Chinese character in the at least one Chinese character; and splice the syllable transition animation at the end of the current Chinese character lip animation to obtain the current lip animation to be fused.

[0222] In the embodiment of the present application, the animation generation module 4554 is further configured to fuse the current lip animation to be fused at a position on the virtual character animation corresponding to the start time of the current syllable pronunciation.

[0223] In the embodiment of the present application, the animation generating device 455 further includes a duration obtaining module 4556 for aligning the audio information and the text information to obtain the syllable pronunciation duration of each Chinese character syllable in the audio information.

[0224] In an embodiment of the present application, the animation generation module 4554 is also used to adjust the animation speed of the Chinese character lip animation based on the syllable pronunciation duration to obtain the lip animation to be merged, thereby obtaining at least one lip animation to be merged corresponding to the at least one Chinese character lip animation; according to the pronunciation sequence corresponding to the at least one Chinese character, based on the start time of the at least one syllable pronunciation, the at least one lip animation to be merged is merged into the virtual character animation to generate the target virtual character animation.

[0225] In the embodiment of the present application, the animation generating device 455 further includes a volume obtaining module 4557 for obtaining the syllable volume of each Chinese character in the audio information from a position on the audio information corresponding to the start time of the syllable pronunciation.

[0226] In an embodiment of the present application, the animation generation module 4554 is also used to use the syllable volume as the playback weight of the Chinese character lip animation, thereby obtaining at least one playback weight corresponding to the at least one Chinese character lip animation; according to the pronunciation sequence corresponding to the at least one Chinese character, based on the at least one syllable pronunciation start time and the at least one playback weight, the at least one Chinese character lip animation is integrated to generate the target virtual character animation.

[0227] In an embodiment of the present application, the animation generation module 4554 is further used to splice the at least one Chinese character lip animation in accordance with the pronunciation sequence corresponding to the at least one Chinese character and based on the at least one syllable pronunciation start time to obtain a target lip animation; determine the earliest syllable pronunciation start time from the at least one syllable pronunciation start time; and fuse the target lip animation at the position on the virtual character animation corresponding to the earliest syllable pronunciation start time to generate the target virtual character animation.

[0228] In an embodiment of the present application, the text alignment module 4552 is further used to perform phoneme splitting on the syllables of each Chinese character in the text information to obtain phoneme information; obtain the phoneme pronunciation start time of each phoneme in the phoneme information in the audio information, thereby obtaining at least one phoneme pronunciation start time corresponding to the phoneme information; determine the target phoneme pronunciation start time corresponding to each Chinese character from the at least one phoneme pronunciation start time corresponding to the phoneme information; and determine the earliest start time of the target phoneme pronunciation start time as the syllable pronunciation start time of each Chinese character.

[0229] In the embodiment of the present application, the animation generating device 455 further includes an animation playing module 4558 for playing the target virtual character animation in a preset virtual scene.

[0230] In an embodiment of the present application, the animation generating device 455 also includes an animation optimization module 4559, which is used to play the target virtual character animation in response to the animation import operation acting on the animation editing client to determine the animation to be optimized in the target virtual character animation; optimize the animation to be optimized to obtain the optimized target virtual character animation.

[0231] In the embodiment of the present application, the animation playing module 4558 is further used to play the optimized target virtual character animation in a preset virtual scene.

[0232] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the animation generation method described in the present invention.

[0233] The embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the animation generation method provided by the embodiment of the present application, for example, Figure 3 The animation generation method shown.

[0234] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0235] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0236] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0237] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0238] To sum up, through the embodiments of the present application, when adjusting the lip shape of the virtual character in the virtual character animation, the lip shape animation is called from the syllable lip shape animation based on each Chinese character in the text information corresponding to the virtual character animation; and the syllable lip shape animation is a lip shape animation based on syllables, and because one Chinese character corresponds to one syllable; therefore, the lip shape animation is integrated based on syllables, which can improve the correspondence effect between the lip shape of the generated animation and the Chinese characters, and therefore, can improve the rendering effect of the generated animation.

[0239] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. An animation generation method, characterized in that: include: Acquire a virtual character animation, wherein the virtual character animation includes audio information and text information; Align the audio information and the text information to obtain the syllable pronunciation start time of each Chinese character in the text information in the audio information; obtain at least one syllable pronunciation start time corresponding to at least one Chinese character in the text information; For each Chinese character, split the syllable of the Chinese character to obtain a Chinese character syllable splitting result, wherein the Chinese character syllable splitting result at least includes the Chinese character syllable final; When the Chinese character syllable splitting result includes the Chinese character syllable final, determining a target initial consonant set corresponding to the Chinese character syllable final from at least one type of initial consonant set, and using the basic initial consonant corresponding to the target initial consonant set as the target basic initial consonant; determining a target final set to which the Chinese character syllable final belongs from at least one type of final set, and using the basic final corresponding to the target final set as the target basic final; Using the lip animation corresponding to the basic syllable formed by the target basic initial consonant and the target basic final as the Chinese character lip animation corresponding to the Chinese character; According to the pronunciation sequence corresponding to the at least one Chinese character, based on the pronunciation start time of the at least one syllable, determine the current Chinese character lip animation corresponding to the current Chinese character from the at least one Chinese character lip animation, wherein the current Chinese character is any one of the at least one Chinese character; When the current Chinese character is a Chinese character other than the last Chinese character among the at least one Chinese character, a syllable transition animation of a preset duration is obtained, wherein the syllable transition animation is obtained based on the lip shape animation of the current Chinese character and the lip shape animation of the next Chinese character, and is a transition animation from the lip shape animation of the current Chinese character to the lip shape animation of the next Chinese character; the preset duration is determined by shortening the duration of the lip shape animation of the current Chinese character and / or the duration of the lip shape animation of the next Chinese character; splicing the syllable transition animation at the end of the current Chinese character lip animation, or splicing the syllable transition animation at the beginning of the next Chinese character lip animation to obtain the current lip animation to be merged; At a position on the virtual character animation corresponding to the current Chinese character, the current lip animation to be merged is merged until the fusion of at least one Chinese character lip animation corresponding to the at least one Chinese character is completed, thereby generating a target virtual character animation; The method further comprises: Based on pronunciation similarity, the initial consonants and finals in the Chinese syllables are classified respectively to obtain at least one set of initial consonants and at least one set of finals; Determine the basic initial consonants corresponding to each type of initial consonant set in the at least one type of initial consonant set, and the basic finals corresponding to each type of final set in the at least one type of final set; Combine the basic initial consonants corresponding to each type of initial consonant set and the basic final consonants corresponding to each type of final consonant set one by one to obtain the basic syllables corresponding to each type of initial consonant set and each type of final consonant set; The lip animation of the basic syllable is drawn to obtain a syllable lip animation, wherein the syllable lip animation includes the lip animation of at least one basic syllable, and the syllable lip animation is a lip animation in units of syllables.

2. The method according to claim 1, characterized in that The method further comprises: When the Chinese character syllable splitting result includes the Chinese character syllable initial consonant and the Chinese character syllable final consonant, determining the target initial consonant set to which the Chinese character syllable initial consonant belongs from the at least one type of initial consonant set, and using the basic initial consonant corresponding to the target initial consonant set as the target basic initial consonant; A target final set to which the Chinese character syllable final belongs is determined from the at least one type of final set, and the basic finals corresponding to the target final set are used as the target basic finals.

3. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Determine the current Chinese character according to the pronunciation sequence corresponding to the at least one Chinese character; Determine the pronunciation start time of the current syllable corresponding to the current Chinese character from the at least one syllable pronunciation start time; Determining a current Chinese character lip shape animation corresponding to the current Chinese character from the at least one Chinese character lip shape animation; At a position on the virtual character animation corresponding to the start time of the current syllable pronunciation, the current Chinese character lip shape animation is merged until the fusion of at least one Chinese character lip shape animation is completed to generate the target virtual character animation.

4. The method according to claim 3, characterized in that The step of fusing the current Chinese character lip shape animation at a position on the virtual character animation corresponding to the start time of the current syllable pronunciation comprises: The current lip animation to be merged is merged at a position on the virtual character animation corresponding to the start time of the current syllable pronunciation.

5. The method according to any one of claims 1 to 3, characterized in that: After obtaining the virtual character animation, the method further includes: Aligning the audio information and the text information to obtain the syllable pronunciation duration of each Chinese character syllable in the audio information; The method further comprises: Based on the syllable pronunciation duration, adjusting the animation speed of the Chinese character lip animation to obtain the lip animation to be merged; obtaining at least one lip animation to be merged corresponding to the at least one Chinese character lip animation; According to the pronunciation sequence corresponding to the at least one Chinese character, based on the pronunciation start time of the at least one syllable, the at least one lip animation to be merged is merged into the virtual character animation to generate the target virtual character animation.

6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: aligning the audio information and the text information to obtain each Chinese character in the text information after the syllable pronunciation start time of the audio information; Acquire the syllable volume of each Chinese character in the audio information from the position on the audio information corresponding to the start time of the syllable pronunciation; The method further comprises: Using the syllable volume as the playback weight of the Chinese character lip animation, obtaining at least one playback weight corresponding to the at least one Chinese character lip animation; According to the pronunciation sequence corresponding to the at least one Chinese character, based on the pronunciation start time of the at least one syllable and the at least one playback weight, the at least one Chinese character lip animation is integrated to generate the target virtual character animation.

7. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: According to the pronunciation sequence corresponding to the at least one Chinese character, based on the pronunciation start time of the at least one syllable, splicing the at least one Chinese character lip animation to obtain a target lip animation; Determining the earliest syllable pronunciation start time from the at least one syllable pronunciation start time; The target lip animation is merged at a position on the virtual character animation corresponding to the start time of the earliest syllable pronunciation to generate the target virtual character animation.

8. The method according to any one of claims 1 to 3, characterized in that: The aligning of the audio information and the text information to obtain the pronunciation start time of the syllable of each Chinese character in the text information in the audio information includes: Perform phoneme splitting on the syllables of each Chinese character in the text information to obtain phoneme information; Acquire the phoneme pronunciation start time of each phoneme in the phoneme information in the audio information, thereby obtaining at least one phoneme pronunciation start time corresponding to the phoneme information; Determine the target phoneme pronunciation start time corresponding to each Chinese character from the at least one phoneme pronunciation start time corresponding to the phoneme information; The earliest start time of the target phoneme pronunciation start time is determined as the syllable pronunciation start time of each Chinese character.

9. The method according to any one of claims 1 to 3, characterized in that: After generating the target virtual character animation, the method further includes: In a preset virtual scene, the target virtual character animation is played.

10. The method according to claim 9, characterized in that Before playing the target virtual character animation in the preset virtual scene, the method further includes: In response to an animation import operation performed on the animation editing client, the target virtual character animation is played to determine an animation to be optimized in the target virtual character animation; Optimizing the animation to be optimized to obtain an optimized target virtual character animation; Playing the target virtual character animation in the preset virtual scene includes: In the preset virtual scene, the optimized target virtual character animation is played.

11. An animation generating device, characterized in that: include: An animation acquisition module, used to acquire virtual character animation, wherein the virtual character animation includes audio information and text information; A text alignment module is used to align the audio information and the text information to obtain the syllable pronunciation start time of each Chinese character in the text information in the audio information; and obtain at least one syllable pronunciation start time corresponding to at least one Chinese character in the text information; The animation calling module is used for splitting the syllable of each Chinese character to obtain a Chinese character syllable splitting result, wherein the Chinese character syllable splitting result at least includes the Chinese character syllable final; when the Chinese character syllable splitting result includes the Chinese character syllable final, determining a target initial consonant set corresponding to the Chinese character syllable final from at least one type of initial consonant set, and using the basic initial consonant corresponding to the target initial consonant set as the target basic initial consonant; determining a target final set to which the Chinese character syllable final belongs from at least one type of final set, and using the basic final corresponding to the target final set as the target basic final; using the lip shape animation corresponding to the basic syllable composed of the target basic initial consonant and the target basic final as the Chinese character lip shape animation corresponding to the Chinese character; The animation generation module is used to determine the current Chinese character lip animation corresponding to the current Chinese character from at least one Chinese character lip animation based on the pronunciation sequence corresponding to the at least one Chinese character and in turn based on the pronunciation start time of the at least one syllable, wherein the current Chinese character is any Chinese character in the at least one Chinese character; when the current Chinese character is a Chinese character other than the last Chinese character in the at least one Chinese character, obtain a syllable transition animation of a preset duration, wherein the syllable transition animation is obtained based on the current Chinese character lip animation and the Chinese character lip animation of the next Chinese character, and is a syllable transition animation from the current Chinese character lip animation to the next Chinese character. The preset duration is determined by shortening the duration of the current Chinese character lip animation and / or the duration of the next Chinese character lip animation; splicing the syllable transition animation at the end of the current Chinese character lip animation, or splicing the syllable transition animation at the beginning of the next Chinese character lip animation to obtain the current lip animation to be merged; fusing the current lip animation to be merged at the position corresponding to the current Chinese character on the virtual character animation until the fusion of at least one Chinese character lip animation corresponding to the at least one Chinese character is completed to generate the target virtual character animation; The lip animation module is used to classify the initials and finals in Chinese syllables based on pronunciation similarity, and obtain at least one set of initial consonants and at least one set of finals; determine the basic initials corresponding to each set of initial consonants in the at least one set of initial consonants, and the basic finals corresponding to each set of finals in the at least one set of finals; combine the basic initials corresponding to each set of initial consonants and the basic finals corresponding to each set of finals one by one, and obtain the basic syllables corresponding to each set of initial consonants and each set of finals; draw the lip animation of the basic syllable to obtain the syllable lip animation, wherein the syllable lip animation includes the lip animation of at least one basic syllable, and the syllable lip animation is a lip animation based on syllables.

12. An animation generating device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 10 when executing the executable instructions stored in the memory.

13. A computer-readable storage medium, characterized in that: Executable instructions are stored, and when executed by a processor, the method described in any one of claims 1 to 10 is implemented.

Images