Method and device for generating virtual character

Abstract

This application provides a method for generating a virtual character, comprising: obtaining a target text; processing the target text according to a text-to-speech (TTS) model to obtain audio features and voice data corresponding to the target text, wherein the TTS model is used to convert the input text into voice data, and the audio features are audio features obtained by the intermediate network layer of the TTS model. Compared to voice data, the audio features carry at least one of richer facial emotion information, pronunciation information, and semantic information; processing the audio features using a pre-trained neural network to obtain facial features of the virtual character; generating a set of image frames of the virtual character based on the facial features, and generating a video including the virtual character based on the set of image frames and the voice data. This application can reduce computational overhead and model deployment costs.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a method and device for generating a virtual character. Background Art

[0002] With the development of artificial intelligence and smart devices, human-computer interaction is becoming increasingly diverse. People are no longer satisfied with the single-dimensional auditory experience and are increasingly integrating text and video playback to enrich the experience. Currently, some virtual character anchors have emerged. Users can enter text on a webpage, and the webpage will generate audio and corresponding virtual character expressions and movements.

[0003] In existing implementations, the voice and facial features of virtual characters are modeled based on text. The text is first converted into speech through text-to-speech (TTS), and then the speech is processed through a neural network to obtain a set of image frames. The two are then fused to generate a video of the virtual character. However, with this method, the virtual character's mouth shape and voice are often not completely aligned, resulting in an unnatural effect. Summary of the Invention

[0004] In a first aspect, the present application provides a method for generating a virtual character, the method comprising:

[0005] Obtain a target text; process the target text according to a text-to-speech (TTS) model to obtain audio features and voice data corresponding to the target text, wherein the TTS model is used to convert the input text into voice data, and the audio features are audio features obtained by the intermediate network layer of the TTS model, and the audio features carry at least one of richer facial emotion information, pronunciation information, and semantic information than the voice data; process the audio features through a neural network to obtain facial features of the virtual character, and the neural network is pre-trained, and the neural network is used to convert the input audio features into facial features; generate an image frame set of the virtual character based on the facial features, and generate a video including the virtual character based on the image frame set and the voice data.

[0006] The audio features obtained by the TTS model when processing the target text carry at least one of richer facial emotion information, pronunciation information, and semantic information than the voice data. In other words, the amount of information and details contained in the audio features are richer. Moreover, relative to the target text, the frame-by-frame audio features are naturally aligned with the image frames. Compared to voice data, these audio feature representations contain a lot of high-order information about voice style. This information can also be passed to facial feature synthesis, achieving complete alignment of sound and expression at the frame level in multiple dimensions such as time frames and emotional style expressions. Furthermore, since the audio features obtained by the TTS model in processing the target text are used as the basis for facial feature extraction, there is no need to re-extract features from the voice data, which reduces computational overhead and model deployment costs, and uses audio features to simultaneously drive voice data prediction and facial feature prediction.

[0007] In a possible implementation, the target text is used for presentation by a virtual character within N frames, the number of the audio features is N, and each of the N audio features corresponds to one frame of the N frames.

[0008] In a possible implementation, processing the target text according to the text-to-speech (TTS) model to obtain audio features includes: obtaining M phonemes of the target text;

[0009] Processing the M phonemes through a neural network to obtain M feature vectors;

[0010] According to the M feature vectors, obtaining, based on an attention mechanism, a correlation degree between each feature vector in the M feature vectors and each frame in the N frames;

[0011] An audio feature of each of the N frames is determined according to the correlation degree and the M feature vectors.

[0012] In a possible implementation, determining the audio feature of each of the N frames according to the correlation degree and the M feature vectors includes:

[0013] According to the correlation degree, a weighted sum is performed on the M feature vectors to determine the audio feature of each frame in the N frames.

[0014] In one possible implementation, the target phoneme is any one of the M phonemes, the target phoneme corresponds to a target feature vector in the M feature vectors, the target phoneme is used for demonstration by the virtual character in multiple frames in N frames, and the target feature vector has the highest correlation with each frame in the multiple frames among the M feature vectors.

[0015] In a possible implementation, the first target frame is any one of the N frames, and obtaining, based on the M feature vectors, a correlation between each feature vector in the M feature vectors and each frame in the N frames based on an attention mechanism includes:

[0016] Acquire speech data of a previous frame adjacent to the first target frame;

[0017] Decoding the speech data of the previous frame adjacent to the first target frame to obtain a decoding state of the previous frame adjacent to the first target frame;

[0018] Based on the attention mechanism, the association degree between each feature vector in the M feature vectors and each frame in the N frames is determined according to the approximation between the decoding state of the previous frame adjacent to the first target frame and each feature vector in the M feature vectors.

[0019] In a possible implementation, the second target frame is any one of the N frames, and the processing of the target text according to the M feature vectors and the text-to-speech (TTS) model to obtain audio features includes:

[0020] Acquire speech data of a previous frame adjacent to the second target frame;

[0021] The speech data of the previous frame adjacent to the second target frame is decoded to obtain a decoding state of the previous frame adjacent to the second target frame, and the decoding state of the previous frame adjacent to the second target frame is used as an audio feature corresponding to the second target frame.

[0022] In a possible implementation, obtaining the target text includes:

[0023] Receive the target text sent by the terminal device.

[0024] In a possible implementation, obtaining the target text includes:

[0025] Displaying a text input box and a text input indication, wherein the target indication is used to instruct the user to input text for presentation by the virtual character in the text input box;

[0026] Get the target text entered by the user in the text input box.

[0027] In a possible implementation, the target text is used for demonstration by a virtual character within N frames, the video includes N frames, and each of the N frames is a frame in the set of image frames.

[0028] In a second aspect, the present application provides a device for generating a virtual character, the device comprising:

[0029] Acquisition module, used to obtain target text;

[0030] A text-to-speech (TTS) module is configured to process the target text according to a text-to-speech (TTS) model to obtain audio features and speech data corresponding to the target text, wherein the TTS model is configured to convert input text into speech data, the audio features are audio features obtained by an intermediate network layer of the TTS model, and the audio features carry at least one of richer facial emotion information, pronunciation information, and semantic information than the speech data;

[0031] A facial feature generation module, configured to process the audio features through a neural network to obtain facial features of the virtual character, wherein the neural network is pre-trained and configured to convert the input audio features into facial features;

[0032] The virtual character generation module is used to generate an image frame set of the virtual character according to the facial features, and to generate a video including the virtual character according to the image frame set and the voice data.

[0033] In a possible implementation, the target text is used for presentation by a virtual character within N frames, the number of the audio features is N, and each of the N audio features corresponds to one frame of the N frames.

[0034] In one possible implementation, the text-to-speech (TTS) module is configured to:

[0035] Obtaining M phonemes of the target text;

[0036] Processing the M phonemes through a neural network to obtain M feature vectors;

[0037] According to the M feature vectors, obtaining, based on an attention mechanism, a correlation degree between each feature vector in the M feature vectors and each frame in the N frames;

[0038] An audio feature of each of the N frames is determined according to the correlation degree and the M feature vectors.

[0039] In one possible implementation, the text-to-speech (TTS) module is configured to:

[0040] According to the correlation degree, a weighted sum is performed on the M feature vectors to determine the audio feature of each frame in the N frames.

[0041] In one possible implementation, the target phoneme is any one of the M phonemes, the target phoneme corresponds to a target feature vector in the M feature vectors, the target phoneme is used for demonstration by the virtual character in multiple frames in N frames, and the target feature vector has the highest correlation with each frame in the multiple frames among the M feature vectors.

[0042] In a possible implementation, the first target frame is any one of the N frames, and the text-to-speech (TTS) module is configured to:

[0043] Acquire speech data of a previous frame adjacent to the first target frame;

[0044] Decoding the speech data of the previous frame adjacent to the first target frame to obtain a decoding state of the previous frame adjacent to the first target frame;

[0045] Based on the attention mechanism, the association degree between each feature vector in the M feature vectors and each frame in the N frames is determined according to the approximation between the decoding state of the previous frame adjacent to the first target frame and each feature vector in the M feature vectors.

[0046] In a possible implementation, the second target frame is any one of the N frames, and the text-to-speech (TTS) module is configured to:

[0047] Acquire speech data of a previous frame adjacent to the second target frame;

[0048] The speech data of the previous frame adjacent to the second target frame is decoded to obtain a decoding state of the previous frame adjacent to the second target frame, and the decoding state of the previous frame adjacent to the second target frame is used as an audio feature corresponding to the second target frame.

[0049] In a possible implementation, the acquisition module is configured to receive a target text sent by a terminal device.

[0050] In a possible implementation, the acquisition module is used to display a text input box and a text input indication, wherein the target indication is used to instruct the user to input text for demonstration by the virtual character in the text input box; and to acquire the target text input by the user in the text input box.

[0051] In a possible implementation, the target text is used for demonstration by a virtual character within N frames, the video includes N frames, and each of the N frames is a frame in the set of image frames.

[0052] In a third aspect, the present application provides an apparatus for generating an avatar, comprising a processor coupled to a memory, the memory storing program instructions. When the processor executes the program instructions stored in the memory, the method described in the first aspect is implemented. For details regarding the steps in each possible implementation of the first aspect performed by the processor, please refer to the first aspect and will not be repeated here.

[0053] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method described in the first aspect above.

[0054] In a fifth aspect, the present application provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute the method described in the first aspect above.

[0055] In a sixth aspect, the present application provides a computer program which, when executed on a computer, enables the computer to execute the method described in the first aspect above.

[0056] In a seventh aspect, the present application provides a chip system, which includes a processor for supporting a server or a threshold value acquisition device to implement the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the server or communication device. The chip system can be composed of a chip or can include a chip and other discrete devices.

[0057] An embodiment of the present application provides a method for generating a virtual character, the method comprising: obtaining a target text; processing the target text according to a text-to-speech (TTS) model to obtain audio features and voice data corresponding to the target text, wherein the TTS model is used to convert the input text into voice data, the audio features are audio features obtained by an intermediate network layer of the TTS model, and the audio features carry at least one of richer facial emotion information, pronunciation information, and semantic information than the voice data; processing the audio features through a neural network to obtain facial features of the virtual character, the neural network being pre-trained and used to convert the input audio features into facial features; generating a set of image frames of the virtual character based on the facial features, and generating a video including the virtual character based on the set of image frames and the voice data. The audio features obtained by the TTS model when processing the target text carry at least one of richer facial emotion information, pronunciation information, and semantic information than the voice data, that is, the audio features contain more information and details. Furthermore, relative to the target text, frame-by-frame audio features are naturally aligned with image frames. Compared to speech data, these audio feature representations contain a wealth of high-level information about speech style, which can also be transferred to facial feature synthesis, achieving complete alignment of sound and expression at the frame level across multiple dimensions, including time frames and emotional style expression. Furthermore, because the audio features obtained during the TTS model's processing of the target text serve as the basis for facial feature extraction, there's no need to re-extract features from the speech data, reducing computational overhead and model deployment costs. Audio features are leveraged to simultaneously drive speech data prediction and facial feature prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 shows an exemplary structural diagram of a communication system;

[0059] Figure 2 shows an exemplary structural diagram of a server;

[0060] Figure 3 An exemplary structural diagram of an electronic device is shown;

[0061] Figure 4 An exemplary structural diagram of a software layer of an electronic device is shown;

[0062] Figure 5 A flowchart of a method for generating a virtual character provided in an embodiment of the present application is provided;

[0063] Figure 6 This is an interface diagram provided in an embodiment of the present application;

[0064] Figure 7 This is an interface diagram provided in an embodiment of the present application;

[0065] Figure 8 This is an interface diagram provided in an embodiment of the present application;

[0066] Figure 9 This is an interface diagram provided in an embodiment of the present application;

[0067] Figure 10 This is an interface diagram provided in an embodiment of the present application;

[0068] Figure 11 A flowchart of a method for generating a virtual character provided in an embodiment of the present application is provided;

[0069] Figure 12 A flowchart of a method for generating a virtual character provided in an embodiment of the present application is provided;

[0070] Figure 13 A flowchart of a method for generating a virtual character provided in an embodiment of the present application is provided;

[0071] Figure 14 A flowchart of a method for generating a virtual character provided in an embodiment of the present application is provided;

[0072] Figure 15 A schematic diagram of a virtual character generation device provided in an embodiment of the present application;

[0073] Figure 16 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0075] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0076] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0077] Figure 1 An exemplary structural diagram of a communication system is shown in FIG. Figure 1 As shown, the communication system includes a server 200 and an electronic device 100. Optionally, the communication system may include one or more servers and each server may include one or more electronic devices within its coverage area, but this application is not limited thereto. Optionally, the communication system may also include other network entities such as a network controller and a switching device, but this application is not limited thereto. Figure 1 The two-way arrow in the figure indicates that there is a communication connection between the server and the electronic device, that is, data can be transmitted between the server and the electronic device through the communication network.

[0078] It should be noted that the above-mentioned communication network can be a local area network, or a wide area network transferred through a relay device, or include a local area network and a wide area network. When the communication network is a local area network, illustratively, the communication network can be a short-range communication network such as a WiFi hotspot network, a WiFi P2P network, a Bluetooth network, a ZigBee network, or a near field communication (NFC) network. When the communication network is a wide area network, illustratively, the communication network can be a third-generation wireless telephone technology (3G) network, a fourth-generation mobile communication technology (4G) network, a fifth-generation mobile communication technology (5G) network, a future-evolved public land mobile network (PLMN) or the Internet, etc., and this application does not limit this.

[0079] In one implementation, the electronic device can obtain the target text input by the user for the virtual character to demonstrate, and the electronic device can send the target text to the server side. The server can generate a video including the virtual character based on the target text. In the video, the virtual character can make a voice corresponding to the target text, and make lip movements, movements, facial expressions, etc. corresponding to the voice. The server can send the video to the electronic device.

[0080] In another implementation, the electronic device can obtain the target text input by the user for demonstration by a virtual character, and generate a video including the virtual character based on the target text. In the video, the virtual character can make a voice corresponding to the target text, and make lip movements, movements, facial expressions, etc. corresponding to the voice.

[0081] It should be understood that Figure 1 The communication system is schematically shown for ease of understanding only, but this does not limit the present application in any way. The communication system may include a greater number of servers and a greater number of electronic devices. The servers communicating with different electronic devices may be the same or different servers, and the number of servers communicating with different electronic devices may be the same or different. This application does not limit this. It should also be understood that the server in the communication system may be any device with transceiver capabilities or a chip that can be provided in the device. Figure 2 An exemplary structural diagram of the server 200 is shown. The structure of the server 200 can be referred to Figure 2 The structure shown.

[0082] The server includes at least one processor 201, at least one memory 202, and at least one network interface 203. The processor 201, memory 202, and network interface 203 are connected, for example, via a bus. In this application, the connection may include various interfaces, transmission lines, or buses, and this embodiment does not limit this. The network interface 203 is used to connect the server to other communication devices via a communication link, such as an Ethernet interface.

[0083] The processor 201 is primarily used to process communication data, control the entire server, execute software programs, and process data from software programs, for example, to support the server in performing the actions described in the embodiments. The processor 201 is primarily used to control the entire server, execute software programs, and process data from software programs. Those skilled in the art will appreciate that a server can include multiple processors to enhance its processing capabilities, and that the various components of the server can be connected via various buses. The processor 201 can also be described as a processing circuit or a processor chip.

[0084] Memory 202 is primarily used to store software programs and data. Memory 202 can exist independently and be connected to processor 201. Alternatively, memory 202 and processor 201 can be integrated together, for example, within a single chip. Memory 202 can store program code for implementing the technical solutions of this application, and its execution is controlled by processor 201. The various computer program codes executed can also be considered drivers for processor 201.

[0085] Figure 2 Only one memory and one processor are shown. In an actual server, multiple processors and multiple memories may exist. Memory may also be referred to as a storage medium or storage device. The memory may be a storage element on the same chip as the processor, i.e., an on-chip storage element, or an independent storage element, which is not limited in this application.

[0086] It should also be understood that the electronic equipment in the communication system can also be referred to as user equipment (UE), which can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted; it can also be deployed on the water (such as ships, etc.); it can also be deployed in the air (for example, on airplanes, balloons and satellites, etc.). Electronic devices can be mobile phones, tablet computers (pads), wearable devices with wireless communication functions (such as smart watches), location trackers with positioning functions, computers with wireless transceiver functions, virtual reality (VR) devices, augmented reality (AR) devices, wireless devices in smart homes, etc., and this application does not limit this. In this application, the aforementioned electronic devices and chips that can be set in the aforementioned electronic devices are collectively referred to as electronic devices.

[0087] The electronic devices in this application may include, but are not limited to: smart mobile phones, televisions, tablet computers, wristbands, head-mounted display devices (HMDs), augmented reality (AR) devices, mixed reality (MR) devices, cellular phones, smart phones, personal digital assistants (PDAs), tablet computers, vehicle-mounted electronic devices, laptop computers, personal computers (PCs), monitoring equipment, robots, vehicle-mounted terminals, autonomous vehicles, etc. Of course, in the following embodiments, there is no limitation on the specific form of the electronic devices.

[0088] For example, see Figure 3 , below, taking a specific structure as an example, the structure of the electronic device provided by this application is exemplarily described.

[0089] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, a motion sensor 180N, etc.

[0090] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0091] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0092] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0093] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0094] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0095] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0096] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0097] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0098] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0099] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0100] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0101] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0102] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0103] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0104] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0105] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0106] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0107] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0108] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0109] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0110] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include, but is not limited to, fifth-generation mobile communication technology (5th-Generation, 5G) system, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long-term evolution (LTE), Bluetooth, global navigation satellite system (GNSS), wireless fidelity (WiFi), nearfield communication (NFC), FM (also known as FM radio), Zigbee protocol, radio frequency identification technology (RFID) and / or infrared (IR) technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite-based augmentation system (SBAS), etc.

[0111] In some embodiments, the electronic device 100 may also include a wired communication module ( Figure 1 ), or the mobile communication module 150 or the wireless communication module 160 here can be replaced with a wired communication module ( Figure 1 (not shown), the wired communication module enables the electronic device to communicate with other devices via a wired network. The wired network may include, but is not limited to, one or more of the following: optical transport network (OTN), synchronous digital hierarchy (SDH), passive optical network (PON), Ethernet, or flexible Ethernet (FlexE).

[0112] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0113] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0114] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0115] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0116] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB camera, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0117] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0118] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0119] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0120] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0121] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0122] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0123] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0124] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0125] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0126] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0127] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0128] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. There are many types of pressure sensors 180A, such as resistive, inductive, and capacitive. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force acts on pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch location based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different touch intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, a command to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, a command to create a new short message is executed.

[0129] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0130] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0131] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0132] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0133] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0134] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0135] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0136] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0137] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0138] The touch sensor 180K is also referred to as a "touch-sensitive device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also referred to as a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0139] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals from the vibrating bones of the human body. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulse signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bones of the human body obtained by the bone conduction sensor 180M to implement voice functions. The application processor can parse heart rate information based on the blood pressure pulse signals obtained by the bone conduction sensor 180M to implement heart rate detection functions.

[0140] The motion sensor 180N can be used to detect moving objects within the range of the camera, and collect the motion profile or motion trajectory of the moving objects. For example, the motion sensor 180N can be an infrared sensor, a laser sensor, a dynamic vision sensor (DVS), etc. The DVS can specifically include sensors such as DAVIS (Dynamic and Active-pixel Vision Sensor), ATIS (Asynchronous Time-based Image Sensor) or CeleX sensor. DVS draws on the characteristics of biological vision. Each pixel simulates a neuron and responds independently to the relative change of light intensity (hereinafter referred to as "light intensity"). When the relative change of light intensity exceeds the threshold, the pixel will output an event signal, including the pixel position, timestamp and characteristic information of the light intensity.

[0141] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0142] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0143] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0144] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0145] The electronic device 100 can be logically divided into a hardware layer, an operating system, and an application layer. The hardware layer includes hardware resources such as the application processor, MCU, memory, modem, Wi-Fi module, sensors, and positioning module described above. This application does not impose any restrictions on the type of operating system installed in the electronic device 100.

[0146] The operating system and application layer can be collectively referred to as the software layer of the electronic device 100. Figure 4 FIG. 1 shows an exemplary structural diagram of the software layer of the electronic device 100. Figure 4 As shown, taking the Android operating system as an example, the operating system is the software middleware between the hardware layer and the application layer, and is a computer program that manages and controls hardware and software resources.

[0147] The application layer includes one or more applications, which can be any type of application, such as social applications, e-commerce applications, browsers, etc. For example, desktop launcher, settings, calendar, camera, photos, calls, and text messages, etc.

[0148] The Android operating system consists of the kernel layer, the Android runtime, system libraries, and the application framework layer. The kernel layer provides low-level system components and services, such as power management, memory management, thread management, and hardware drivers. Hardware drivers include display drivers, camera drivers, audio drivers, and touch drivers. The kernel layer encapsulates the kernel drivers and provides interfaces to the application framework layer, shielding them from lower-level implementation details.

[0149] The Android runtime and system libraries provide the library files and execution environment required by executable programs at runtime. A virtual machine or virtual machine instance converts application bytecode into machine code. System libraries provide runtime support for executable programs and include a 2D graphics engine, 3D graphics engine, media library, surface manager, and state monitoring services.

[0150] The application framework layer is used to provide various basic common components and services for applications in the application layer, including window manager, activity manager, package manager, resource manager, display policy service, etc.

[0151] The functions of the various components of the operating system described above can be implemented by the application processor executing the program stored in the memory.

[0152] Those skilled in the art will appreciate that the electronic device 100 may include Figure 3 Fewer or more components than shown, Figure 3 The electronic device shown only includes components that are more relevant to the various implementations disclosed in this application.

[0153] Reference Figure 5 , Figure 5 The process diagram of a method for generating a virtual character provided in an embodiment of the present application is as follows: Figure 5 As shown, the virtual character generation method provided in the embodiment of the present application includes:

[0154] 501. Get the target text.

[0155] The execution subject of step 501 may be an electronic device. Specifically, the user may input a target text for the virtual character to demonstrate on the electronic device, and accordingly, the electronic device may obtain the target text.

[0156] The execution subject of step 501 may be a server. Specifically, the user may input a target text for the virtual character to demonstrate on the electronic device, and the electronic device may send the target text to the server. Accordingly, the server may obtain the target text.

[0157] In an embodiment of the present application, the electronic device can display a text input box and a text input indication, wherein the target indication is used to instruct the user to input text for demonstration by the virtual character in the text input box, and obtain the target text input by the user in the text input box.

[0158] In one scenario, an electronic device may be installed with an application that can generate a video including a virtual character based on a target text. The user can open the relevant application and enter the target text for the virtual character to demonstrate in the application. The electronic device can then generate a video including the virtual character based on the target text, or send the target text to a server, and the server will generate a video including the virtual character based on the target text.

[0159] Take the mobile phone as an example, for details, please refer to Figure 6 ,The mobile phone can be installed with an application that can ,generate a video including a virtual character based on the target text, ,e.g. Figure 6 The user can open the virtual character APP, and in response to the user's click operation, the mobile phone can display the following Figure 7 The interface shown in FIG. 1 may include, but is not limited to, a prompt for instructing the user to input text, an input box for the user to input a target text, and related controls such as “OK”. The user may input the target text (e.g., Figure 7 The target text entered by the user is "Who am I?"), and then the user clicks the "OK" control, and then the mobile phone can generate and display a video including a virtual character according to the target text, or send the target text to the server, and the server generates a video including a virtual character according to the target text, and the mobile phone can receive the video including the virtual character sent by the server and display the video. For example, you can refer to Figure 8 , the mobile phone can display a video including a virtual character, where the virtual character can speak the voice corresponding to the target text, such as Figure 8 As shown, the virtual character can say "Who am I", and at the same time, the virtual character can make lip movements, facial expressions, etc. corresponding to the voice.

[0160] It should be understood that the above-mentioned "lip shapes corresponding to the voice, etc." can be understood as the lip shapes that a person needs to make when speaking a certain voice correctly and clearly, and "facial expressions corresponding to the voice, etc." can be understood as the expressions corresponding to the emotions contained in the voice when a person speaks a certain voice correctly and clearly. For example, if the voice contains angry emotions, a person can show an angry facial expression (such as frowning, etc.) while speaking the voice.

[0161] In another scenario, developers can configure a web page with the function of generating a virtual character based on the target text. Users can open the web page on an electronic device and enter the target text in the web page. The web page can then generate a video including the virtual character based on the target text for users to preview on the web page or download to the electronic device.

[0162] Taking the electronic device as a portable computer as an example, for details, please refer to Figure 9 ,The user can open a web page on a portable computer that has the function of generating a virtual character based on the target text, e.g. Figure 9 The web page shown in the figure may include, but is not limited to, a prompt for the user to input text, an input box for the user to input a target text, and related controls such as "OK". The user can enter the target text (e.g., Figure 9 The target text entered by the user is "Who am I?"), and then the user can click the "OK" control, and then the portable computer can send the target text to the server, and the server generates a video including a virtual character according to the target text. The mobile phone can receive the video including the virtual character sent by the server and display the video or instruct the user to perform subsequent operations on the food (preview or download). For example, you can refer to Figure 10 The web page may display a prompt instructing the user to perform subsequent operations on the food (preview or download), and the user may click the "preview" control. Accordingly, the web page may display a preview of the video on the page, and the user may click the "download" control. Accordingly, the portable computer may save the video locally.

[0163] In another scenario, the user can make a sound at the electronic device, and the electronic device can recognize the target text expressed by the user's voice, and then generate a video including a virtual character based on the target text, or send the target text to the server, and the server generates a video including the virtual character based on the target text.

[0164] 502. Process the target text according to a text-to-speech (TTS) model to obtain audio features and voice data corresponding to the target text, wherein the TTS model is used to convert the input text into voice data, and the audio features are audio features obtained by the intermediate network layer of the TTS model, and the audio features carry at least one of richer facial emotion information, pronunciation information, and semantic information than the voice data.

[0165] The execution entity of step 502 may be an electronic device or a server.

[0166] In one implementation, after obtaining the target text input by the user, the electronic device can process the target text according to the text-to-speech TTS model to obtain audio features and voice data for the virtual character demonstration.

[0167] In one implementation, after obtaining the target text input by the user, the electronic device can send the target text to the server. The server can process the target text according to the text-to-speech TTS model to obtain audio features and voice data for virtual character demonstration.

[0168] Taking the execution subject of step 502 as a server as an example, in an embodiment of the present application, after obtaining the target text, the server can first pre-process the target text and process the target text into a sequence adapted to the input format of the TTS model. For example, the server can normalize the target text, convert the non-standard target text into a pronounceable format, and perform word segmentation processing, dividing the sentences in the target text by word units to resolve sentence ambiguity, and perform prosodic analysis to predict the pause rhythm and / or stress of each sentence in the target text, etc., and convert the characters of the target text to the phoneme level to obtain a phoneme string, and convert the phoneme string into the sequence format required by the TTS model (subsequent embodiments may refer to it as an ID sequence).

[0169] Specifically, the server can obtain M phonemes of the target text and process the M phonemes through a neural network to obtain M feature vectors. Figure 11 , the server can first convert the target text into a serialized ID sequence, where each identifier in the ID sequence can correspond to one of the M phonemes, and then use a neural network (e.g. Figure 11 The convolutional neural networks (CNN), recurrent neural networks (RNN), transformer and other network structures shown in the figure encode the ID sequence into M feature vectors (or embedding vectors), where the feature vectors can represent the abstract content information of the target text.

[0170] In an embodiment of the present application, the TTS model can convert the target text into acoustic features, generate voice data through the acoustic features, and output the audio features (or hidden features) obtained in the process of processing the target text, wherein the hidden features carry richer high-dimensional features (for example, but not limited to at least one of facial emotion information, pronunciation information, and semantic information) compared to the voice data.

[0171] In an embodiment of the present application, the server may, based on the M feature vectors, obtain a correlation between each feature vector in the M feature vectors and each frame in the N frames based on an attention mechanism; and determine the audio features of each frame in the N frames based on the correlation and the M feature vectors. More specifically, the server may perform a weighted summation of the M feature vectors based on the correlation to determine the audio features of each frame in the N frames.

[0172] In an embodiment of the present application, the first target frame is any frame among the N frames, and the server can obtain the voice data of the previous frame adjacent to the first target frame; decode the voice data of the previous frame adjacent to the first target frame to obtain the decoding state of the previous frame adjacent to the first target frame, and based on the attention mechanism, determine the correlation between each feature vector in the M feature vectors and each frame in the N frames according to the approximation between the decoding state of the previous frame adjacent to the first target frame and each feature vector in the M feature vectors.

[0173] Reference Figure 12 , the first target frame can be Figure 12 As shown in T, the server can obtain the voice data of the previous frame (T-1 frame) adjacent to the T frame (for example, it can be Figure 12 The Mel spectrum Mel shown in ), and Mel is input into the Pre-net module and the decoding module to obtain the decoding state Decoder state. After that, the server can determine the correlation between each feature vector in the M feature vectors and each frame in the N frames according to the approximation between the decoding state and each feature vector in the M feature vectors, and perform weighted summation on the M feature vectors to determine the audio features of each frame in the N frames ( Figure 12 The server can also determine the voice data Mel of the first target frame based on the audio feature Context T and the decoding state Decoderstate. It should be understood that the voice feature is not limited to Figure 12 The Mel shown in the figure may also be a bark spectrum, etc., which is not limited in this application.

[0174] Next, it is described how the server determines the degree of association between each feature vector in the M feature vectors and each frame in the N frames according to the degree of approximation between the decoding state and each feature vector in the M feature vectors.

[0175] In an embodiment of the present application, the server may obtain the decoding state of the previous frame adjacent to the first target frame, and use the similarity between the decoding state and M feature vectors (each feature vector corresponds to one frame) as the correlation between each feature vector in the M feature vectors and each frame in the N frames. In an optional implementation, the correlation may be calculated by the following calculation process:

[0176] e=v T tanh(Query*s+Key*h+U);

[0177] α=softmax(e);

[0178] Among them, tanh is the hyperbolic tangent function; Softmax is the normalized exponential function, which compresses each element in e to (0,1) and the sum is 1, so as to serve as a weighting coefficient; s, h, U, and v are all trainable weight coefficients in the attention mechanism Atttention network.

[0179] The larger the correlation degree α value is, the closer the feature vector Embedding is to the decoding Decoder state. It can be considered that the phoneme is currently being decoded. Then, the M feature vectors can be weighted and summed according to the correlation degree α to determine the audio feature Context of each frame in the N frames.

[0180] context=∑ j α j *Key j ;

[0181] For example, please refer to Figure 13 , Decoder state represents the decoding state of the previous frame, Embedding(i) represents the i-th feature vector, and it is necessary to calculate the similarity (α1, α2, ..., α) between the decoding state of the previous frame and each feature vector Embedding(i) j ), and according to the similarity (α1, α2, ..., α j ) performs weighted summation on the feature vector Embedding(i) to obtain the audio feature Context of the current frame.

[0182] In an embodiment of the present application, in addition to using the Context in the above embodiment as the audio feature output during the TTS model processing the target text, the decoding status of each frame can also be used as the audio feature output during the TTS model processing the target text. Specifically, the second target frame is any frame among the N frames. The server can obtain the voice data of the previous frame adjacent to the second target frame, and decode the voice data of the previous frame adjacent to the second target frame to obtain the decoding status of the previous frame adjacent to the second target frame. The decoding status of the previous frame adjacent to the second target frame is used as the audio feature corresponding to the second target frame.

[0183] In an embodiment of the present application, the above-mentioned audio feature Context and the decoding status may be weighted or other operations may be performed on the audio feature Context and the decoding status to obtain the audio features of each frame, which is not limited in the embodiment of the present application.

[0184] In a possible implementation, the target text is used for presentation by a virtual character within N frames, the number of the audio features is N, and each of the N audio features corresponds to one frame of the N frames.

[0185] In one possible implementation, the target phoneme is any one of the M phonemes, the target phoneme corresponds to a target feature vector in the M feature vectors, the target phoneme is used for demonstration by the virtual character in multiple frames out of N frames, and the target feature vector has the highest correlation with each of the multiple frames among the M feature vectors. In other words, for the target phoneme for demonstration by the virtual character in multiple frames, within these multiple frames, the correlation between the target phoneme and the target feature vector is the highest among the correlations between the target phoneme and all the feature vectors, so that the audio features can be fully aligned with the factors of the target text.

[0186] In this embodiment of the present application, in addition to obtaining the audio features generated by the TTS model during the processing of the target text, the server also needs to obtain voice data, which can be the Mel spectrum or Bark spectrum in the above embodiment. The server can synthesize the voice data into a human-recognizable sound through a vocoder.

[0187] 503. Process the audio features through a neural network to obtain facial features of the virtual character, where the neural network is pre-trained and is used to convert the input audio features into facial features.

[0188] The execution entity of step 503 may be an electronic device or a server. Taking the server as an example, the server may process the audio features through a neural network to obtain the facial features of the virtual character.

[0189] Specifically, the server can map the audio features to facial features, and render and reconstruct the character image based on the mapped facial features, where the facial features can be commonly used 2D lip shape and / or eye key points, 3D blend shape coefficients, etc.

[0190] 504. Generate an image frame set of the virtual character based on the facial features, and generate a video including the virtual character based on the image frame set and the voice data.

[0191] In this embodiment of the present application, the server may generate a set of image frames of the avatar based on the facial features, convert the image frames into a video, and simultaneously fuse the video with the voice data acquired in step 502 to generate a video including the avatar. The target text is used for presentation by the avatar within N frames, and the video includes N frames, each of which is a frame in the set of image frames.

[0192] For more details, please refer to Figure 14 In the process of generating virtual characters, the text can be preprocessed first, and the preprocessed text can be encoded to obtain the feature vector Embedding. Then, alignment is performed based on the attention mechanism, and the audio feature Context is obtained. Based on the audio features, facial features (facial parameters & image reconstruction) are extracted and sound is generated. Then, video is synthesized based on the facial features and sound.

[0193] In the embodiment of the present application, since both the voice data and facial features are obtained based on the audio features, the audio and video can be fully aligned, so that the voice and the facial features of the virtual character are accurately matched. At the same time, the amount of information in the audio features obtained by the TTS model when processing the target text gradually increases compared to the target text, and the amount of information and details contained in the audio features are richer. Relative to the target text, the frame-by-frame audio features are naturally aligned with the image frames. Compared to the voice data, these audio feature representations contain a lot of high-order information about the voice style, which can also be losslessly transferred to the facial feature synthesis, achieving full alignment of the sound and expression in multiple dimensions such as time frame and emotional style expression at the frame level.

[0194] In the embodiment of the present application, since the audio features obtained during the TTS model processing of the target text are used as the basis for facial feature extraction, there is no need to re-extract features from the voice data, which reduces the computational overhead and model deployment cost, and utilizes audio features to simultaneously drive voice data prediction and facial feature prediction.

[0195] It should be understood that during the training process of the TTS model, the voice data in the video can be used to guide the TTS model alignment and generate corresponding audio features. Since the decoding Decoder in the TTS model is an iterative process, the errors generated in the process of predicting the voice data will continue to accumulate. Therefore, the voice data in the video can be extracted first, and the audio features corresponding to the trained TTS model can be extracted. Then, this feature can be sent to the Pre-net / Decoder network to guide the alignment process in this way.

[0196] An embodiment of the present application provides a method for generating a virtual character, the method comprising: obtaining a target text; processing the target text according to a text-to-speech (TTS) model to obtain audio features and voice data corresponding to the target text, wherein the TTS model is used to convert the input text into voice data, the audio features are audio features obtained by an intermediate network layer of the TTS model, and the audio features carry at least one of richer facial emotion information, pronunciation information, and semantic information than the voice data; processing the audio features through a neural network to obtain facial features of the virtual character, the neural network being pre-trained and used to convert the input audio features into facial features; generating a set of image frames of the virtual character based on the facial features, and generating a video including the virtual character based on the set of image frames and the voice data. The audio features obtained by the TTS model when processing the target text carry at least one of richer facial emotion information, pronunciation information, and semantic information than the voice data, that is, the audio features contain more information and details. Furthermore, relative to the target text, frame-by-frame audio features are naturally aligned with image frames. Compared to speech data, these audio feature representations contain a wealth of high-level information about speech style, which can be losslessly transferred to facial feature synthesis, achieving complete alignment of sound and expression at the frame level across multiple dimensions, including time frames and emotional expression. Furthermore, because the audio features obtained from the TTS model processing the target text serve as the basis for facial feature extraction, there's no need to re-extract features from the speech data, reducing computational overhead and model deployment costs. Audio features are leveraged to simultaneously drive speech data prediction and facial feature prediction.

[0197] Reference Figure 15 The embodiment of the present application further provides a virtual character generation device 1500, the device comprising:

[0198] An acquisition module 1501 is used to acquire a target text;

[0199] A text-to-speech (TTS) module 1502 is configured to process the target text according to a text-to-speech (TTS) model to obtain audio features and speech data corresponding to the target text, wherein the TTS model is configured to convert input text into speech data, and the audio features are audio features obtained by an intermediate network layer of the TTS model, and the audio features carry at least one of richer facial emotion information, pronunciation information, and semantic information than the speech data;

[0200] A facial feature generation module 1503 is configured to process the audio features through a neural network to obtain facial features of the virtual character, wherein the neural network is pre-trained and configured to convert the input audio features into facial features;

[0201] The virtual character generation module 1504 is configured to generate a set of image frames of the virtual character based on the facial features, and generate a video including the virtual character based on the set of image frames and the voice data.

[0202] In a possible implementation, the target text is used for presentation by a virtual character within N frames, the number of the audio features is N, and each of the N audio features corresponds to one frame of the N frames.

[0203] In one possible implementation, the text-to-speech (TTS) module is configured to:

[0204] Obtaining M phonemes of the target text;

[0205] Processing the M phonemes through a neural network to obtain M feature vectors;

[0206] According to the M feature vectors, obtaining, based on an attention mechanism, a correlation degree between each feature vector in the M feature vectors and each frame in the N frames;

[0207] An audio feature of each of the N frames is determined according to the correlation degree and the M feature vectors.

[0208] In one possible implementation, the text-to-speech (TTS) module 1502 is configured to:

[0209] According to the correlation degree, a weighted sum is performed on the M feature vectors to determine the audio feature of each frame in the N frames.

[0210] In one possible implementation, the target phoneme is any one of the M phonemes, the target phoneme corresponds to a target feature vector in the M feature vectors, the target phoneme is used for demonstration by the virtual character in multiple frames in N frames, and the target feature vector has the highest correlation with each frame in the multiple frames among the M feature vectors.

[0211] In a possible implementation, the first target frame is any one of the N frames, and the text-to-speech (TTS) module is configured to:

[0212] Acquire speech data of a previous frame adjacent to the first target frame;

[0213] Decoding the speech data of the previous frame adjacent to the first target frame to obtain a decoding state of the previous frame adjacent to the first target frame;

[0214] Based on the attention mechanism, the association degree between each feature vector in the M feature vectors and each frame in the N frames is determined according to the approximation between the decoding state of the previous frame adjacent to the first target frame and each feature vector in the M feature vectors.

[0215] In a possible implementation, the second target frame is any one of the N frames, and the text-to-speech (TTS) module is configured to:

[0216] Acquire speech data of a previous frame adjacent to the second target frame;

[0217] The speech data of the previous frame adjacent to the second target frame is decoded to obtain a decoding state of the previous frame adjacent to the second target frame, and the decoding state of the previous frame adjacent to the second target frame is used as an audio feature corresponding to the second target frame.

[0218] In a possible implementation, the acquisition module 1501 is configured to receive a target text sent by a terminal device.

[0219] In a possible implementation, the acquisition module 1501 is used to display a text input box and a text input indication, where the target indication is used to instruct the user to input text for the virtual character to demonstrate in the text input box; and to obtain the target text input by the user in the text input box.

[0220] In a possible implementation, the target text is used for demonstration by a virtual character within N frames, the video includes N frames, and each of the N frames is a frame in the set of image frames.

[0221] The description related to the acquisition module 1501 can refer to the description of step 501, and the similarities are not repeated here.

[0222] The description related to the text-to-speech TTS module 1502 can refer to the description of step 502, and the similarities are not repeated here.

[0223] The description related to the facial feature generation module 1503 can refer to the description of step 503, and the similarities are not repeated here.

[0224] The description related to the virtual character generation module 1504 can refer to the description of step 504, and the similarities are not repeated here.

[0225] In a simple embodiment, those skilled in the art will appreciate that the electronic device may be configured to Figure 16 The form shown.

[0226] like Figure 16 The device 1600 shown includes at least one processor 1601 , a transceiver 1602 , and optionally, a memory 1603 .

[0227] In a possible implementation, the device 1600 may further include a display 1606 ; the device may further include a sensor 1605 for capturing the posture and position of the electronic device.

[0228] Memory 1603 may be a volatile memory, such as a random access memory; the memory may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1603 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1603 may be a combination of the above memories.

[0229] The specific connection medium between the processor 1601 and the memory 1603 is not limited in the embodiment of the present application. In the figure, the memory 1603 and the processor 1601 are connected via a bus. The bus is represented by a thick line in the figure. The connection between other components is only for schematic illustration and is not limited. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 16 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0230] When the electronic device uses Figure 16 When the form shown is Figure 16 The processor 1601 in the embodiment can call the computer execution instructions stored in the memory 1603, so that the electronic device can execute the method executed by the electronic device in any of the above method embodiments.

[0231] When the processor 1601 performs a function of triggering display, such as an operation involving displaying an image or video, the processor 1601 may display a video including a virtual character through the display 1606 in the device 1600 .

[0232] Optionally, when executing the display function, the processor 1601 may also display images or videos through displays in other devices, such as sending display instructions to other devices to instruct them to display videos including virtual characters.

[0233] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0234] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0235] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0236] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0237] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, server, or other network device, etc.) to execute the present application. Figures 2 to 16 The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., which can store program code.

[0238] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a virtual character, characterized in that: The method comprises: Get the target text; Processing the target text according to a text-to-speech (TTS) model to obtain audio features and voice data corresponding to the target text, wherein the TTS model is used to convert the input text into voice data, the audio features are audio features obtained by an intermediate network layer of the TTS model, the audio features are latent layer features, and the audio features carry at least one of richer facial emotion information, pronunciation information, and semantic information than the voice data; Processing the audio features through a neural network to obtain facial features of the virtual character, wherein the neural network is pre-trained and is used to convert the input audio features into facial features; A set of image frames of the virtual character is generated based on the facial features, and a video including the virtual character is generated based on the set of image frames and the voice data.

2. The method according to claim 1, characterized in that The target text is used for demonstration by a virtual character within N frames, the number of the audio features is N, and each of the N audio features corresponds to one frame in the N frames.

3. The method according to claim 2, characterized in that The processing of the target text according to the text-to-speech (TTS) model to obtain audio features includes: Obtaining M phonemes of the target text; Processing the M phonemes through a neural network to obtain M feature vectors; According to the M feature vectors, obtaining, based on an attention mechanism, a correlation degree between each feature vector in the M feature vectors and each frame in the N frames; An audio feature of each of the N frames is determined according to the correlation degree and the M feature vectors.

4. The method according to claim 3, characterized in that Determining the audio feature of each frame in the N frames according to the correlation degree and the M feature vectors includes: According to the correlation degree, a weighted sum is performed on the M feature vectors to determine the audio feature of each frame in the N frames.

5. The method according to claim 3, characterized in that The target phoneme is any one of the M phonemes, the target phoneme corresponds to a target feature vector in the M feature vectors, the target phoneme is used for demonstration by the virtual character in multiple frames in N frames, and the target feature vector has the greatest correlation with each frame in the multiple frames among the M feature vectors.

6. The method according to any one of claim 3, characterized in that: The first target frame is any one of the N frames, and obtaining, based on the M feature vectors, a correlation degree between each feature vector in the M feature vectors and each frame in the N frames based on an attention mechanism includes: Acquire speech data of a previous frame adjacent to the first target frame; Decoding the speech data of the previous frame adjacent to the first target frame to obtain a decoding state of the previous frame adjacent to the first target frame; Based on the attention mechanism, the association degree between each feature vector in the M feature vectors and each frame in the N frames is determined according to the approximation between the decoding state of the previous frame adjacent to the first target frame and each feature vector in the M feature vectors.

7. The method according to claim 2, characterized in that The second target frame is any one of the N frames, and the processing of the target text according to the text-to-speech (TTS) model to obtain audio features includes: Acquire speech data of a previous frame adjacent to the second target frame; The speech data of the previous frame adjacent to the second target frame is decoded to obtain a decoding state of the previous frame adjacent to the second target frame, and the decoding state of the previous frame adjacent to the second target frame is used as an audio feature corresponding to the second target frame.

8. The method according to any one of claims 1 to 7, characterized in that: The obtaining of the target text includes: Receive the target text sent by the terminal device.

9. The method according to any one of claims 1 to 7, characterized in that: The obtaining of the target text includes: Displaying a text input box and a text input instruction, wherein the text input instruction is used to instruct the user to input text for presentation by the virtual character in the text input box; Get the target text entered by the user in the text input box.

10. The method according to any one of claims 1 to 7, characterized in that: The target text is used for demonstration by a virtual character within N frames, the video includes N frames, and each of the N frames is a frame in the image frame set.

11. A virtual character generating device, characterized in that: The device comprises: Acquisition module, used to obtain target text; A text-to-speech (TTS) module is configured to process the target text according to a text-to-speech (TTS) model to obtain audio features and speech data corresponding to the target text, wherein the TTS model is configured to convert input text into speech data, the audio features are audio features obtained by an intermediate network layer of the TTS model, the audio features are latent features, and the audio features carry at least one of richer facial emotion information, pronunciation information, and semantic information than the speech data; A facial feature generation module, configured to process the audio features through a neural network to obtain facial features of the virtual character, wherein the neural network is pre-trained and configured to convert the input audio features into facial features; The virtual character generation module is used to generate an image frame set of the virtual character according to the facial features, and to generate a video including the virtual character according to the image frame set and the voice data.

12. The device according to claim 11, characterized in that The target text is used for demonstration by a virtual character within N frames, the number of the audio features is N, and each of the N audio features corresponds to one frame in the N frames.

13. The device according to claim 12, characterized in that The text-to-speech (TTS) module is used to: Obtaining M phonemes of the target text; Processing the M phonemes through a neural network to obtain M feature vectors; According to the M feature vectors, obtaining, based on an attention mechanism, a correlation degree between each feature vector in the M feature vectors and each frame in the N frames; An audio feature of each of the N frames is determined according to the correlation degree and the M feature vectors.

14. The device according to claim 13, characterized in that The text-to-speech (TTS) module is used to: According to the correlation degree, a weighted sum is performed on the M feature vectors to determine the audio feature of each frame in the N frames.

15. The device according to claim 13, characterized in that The target phoneme is any one of the M phonemes, the target phoneme corresponds to a target feature vector in the M feature vectors, the target phoneme is used for demonstration by the virtual character in multiple frames in N frames, and the target feature vector has the greatest correlation with each frame in the multiple frames among the M feature vectors.

16. The device according to any one of claim 13, characterized in that: The first target frame is any one of the N frames, and the text-to-speech TTS module is used to: Acquire speech data of a previous frame adjacent to the first target frame; Decoding the speech data of the previous frame adjacent to the first target frame to obtain a decoding state of the previous frame adjacent to the first target frame; Based on the attention mechanism, the association degree between each feature vector in the M feature vectors and each frame in the N frames is determined according to the approximation between the decoding state of the previous frame adjacent to the first target frame and each feature vector in the M feature vectors.

17. The device according to claim 12, characterized in that The second target frame is any one of the N frames, and the text-to-speech TTS module is used to: Acquire speech data of a previous frame adjacent to the second target frame; The speech data of the previous frame adjacent to the second target frame is decoded to obtain a decoding state of the previous frame adjacent to the second target frame, and the decoding state of the previous frame adjacent to the second target frame is used as an audio feature corresponding to the second target frame.

18. The device according to any one of claims 11 to 17, characterized in that The acquisition module is used to receive the target text sent by the terminal device.

19. The device according to any one of claims 11 to 17, characterized in that The acquisition module is used to display a text input box and a text input instruction, wherein the text input instruction is used to instruct the user to input text for the virtual character demonstration in the text input box; and to acquire the target text input by the user in the text input box.

20. The device according to any one of claims 11 to 17, characterized in that The target text is used for demonstration by a virtual character within N frames, the video includes N frames, and each of the N frames is a frame in the image frame set.

21. A virtual character generation device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, and the computer program comprises program instructions. When the processor runs the program instructions, the processor executes the method according to any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program request is executed by a computer, the computer is caused to execute the method according to any one of claims 1 to 10.

Images