Video generation method, device, electronic device and storage medium

Abstract

The present disclosure provides a video generation method, which relates to the field of artificial intelligence technology, especially to the technical fields of augmented reality, virtual reality, computer vision, deep learning, etc., and can be applied to scenes such as the metaverse and virtual digital people. The specific implementation scheme is: according to the target action driving coefficient among multiple action driving coefficients, the target virtual image is rendered using a graphics processing unit to obtain a target image; in response to obtaining the target image, the target image is encoded using a graphics processing unit to obtain target video sequence data; according to the previous video sequence data and the target video sequence data at the target moment, the current video data is obtained, wherein the previous video sequence data at the target moment corresponds to the previous time period of the target moment; and in response to determining that the target moment meets the preset conditions, the target video is obtained according to the current video data. The present disclosure also provides a video generation device, an electronic device, and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the fields of augmented reality, virtual reality, computer vision, deep learning, etc., and can be applied to scenarios such as the metaverse and virtual digital humans. More specifically, the present disclosure provides a video generation method, device, electronic device, and storage medium. Background Art

[0002] With the development of artificial intelligence technology, the application scenarios of virtual images are increasing. The virtual image can be rendered to generate a video. In the video, the virtual image can perform multiple actions. Summary of the invention

[0003] The present disclosure provides a video generation method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, a video generation method is provided, the method comprising: rendering a target virtual image using a target computing unit according to a target action driving coefficient among a plurality of action driving coefficients to obtain a target image, wherein the plurality of action driving coefficients respectively correspond to a plurality of moments, and the target action driving coefficient corresponds to a target moment among the plurality of moments; in response to obtaining the target image, encoding the target image using a target computing unit to obtain target video sequence data; obtaining current video data according to previous video sequence data and target video sequence data at the target moment, wherein the previous video sequence data at the target moment corresponds to a previous time period of the target moment, and the previous time period of the target moment includes at least one moment before the target moment; and in response to determining that the target moment meets a preset condition, obtaining a target video according to the current video data.

[0005] According to another aspect of the present disclosure, a video generating device is provided, which includes: a rendering module, which is used to render a target virtual image using a target calculation unit according to a target action driving coefficient among multiple action driving coefficients to obtain a target image, wherein the multiple action driving coefficients correspond to multiple moments respectively, and the target action driving coefficient corresponds to a target moment among the multiple moments; an encoding module, which is used to encode the target image using a target calculation unit in response to obtaining the target image to obtain target video sequence data; a first acquisition module, which is used to obtain current video data based on previous video sequence data and target video sequence data at the target moment, wherein the previous video sequence data at the target moment corresponds to a previous time period of the target moment, and the previous time period of the target moment includes at least one moment before the target moment; and a second acquisition module, which is used to obtain a target video based on current video data in response to determining that the target moment meets a preset condition.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram of an exemplary system architecture to which a video generation method and apparatus can be applied according to an embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of a video generation method according to an embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of a video generation method according to another embodiment of the present disclosure;

[0014] Figure 4 is a flowchart of a video generation method according to another embodiment of the present disclosure;

[0015] Figure 5 is a schematic diagram of an interactive interface according to an embodiment of the present disclosure;

[0016] Figure 6 is a schematic diagram of a video frame according to an embodiment of the present disclosure;

[0017] Figure 7 is a block diagram of a video generating apparatus according to an embodiment of the present disclosure; and

[0018] Figure 8 is a block diagram of an electronic device to which a video generating method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] A virtual image can be used to broadcast text to generate a related video. For example, the broadcast text can be displayed on an interactive interface. For another example, a voice-to-animation (VTA) algorithm can be used to process the broadcast text to obtain a processing result. The processing result may include a timestamp of each character in the broadcast text and at least one action drive coefficient corresponding to each character. The action drive coefficient may be a blend shape (BS) coefficient. The action drive coefficient may be applied to multiple blend shape bases of the virtual image's face, so that the shapes of these blend shape bases change so that the virtual image performs corresponding expression actions.

[0021] In some embodiments, according to the speech animation synthesis technology, the central processing unit (CPU) can be used to process the broadcast text to obtain a processing result. According to the action driving coefficient in the processing result, the virtual image is rendered using a rendering engine deployed on the graphics processing unit (GPU), and multiple images can be obtained. These images can be stored in the memory or hard disk. The central processing unit can read these images from the memory or disk for video encoding. However, these images occupy a large storage space, and the time required for the central processing unit to read these images is also long, resulting in low video generation efficiency. For example, for a segment of broadcast text, thousands or tens of thousands of images may be rendered, which may occupy dozens of gigabytes (Gb) of storage space.

[0022] Figure 1 is a schematic diagram of an exemplary system architecture to which a video generation method and apparatus can be applied according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0023] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0024] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptops, desktop computers, etc.

[0025] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0026] It should be noted that the video generation method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the video generation device provided in the embodiment of the present disclosure can generally be set in the server 105. The video generation method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the video generation device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0027] Figure 2 is a flowchart of a video generation method according to an embodiment of the present disclosure.

[0028] like Figure 2 As shown, the method 200 may include operations S210 to S240.

[0029] In operation S210, a target virtual image is rendered using a target calculation unit according to a target action driving coefficient among the plurality of action driving coefficients to obtain a target image.

[0030] In the embodiment of the present disclosure, the plurality of action driving coefficients correspond to the plurality of moments respectively, and the target action driving coefficient corresponds to the target moment among the plurality of moments. For example, the plurality of action driving coefficients may be obtained by processing the target text using a preset algorithm. The target text may be the above-mentioned broadcast text.

[0031] In the disclosed embodiment, the target text may be input by the user. For example, the target text Text1 may be “Hello, welcome to experience the speech animation synthesis technology”.

[0032] In the disclosed embodiment, the target action driving coefficient may be any one of the plurality of action driving coefficients. For example, the target action driving coefficient may correspond to a character in the target text. It is understood that a character in the target text may correspond to at least one action driving coefficient.

[0033] In the disclosed embodiments, the target computing unit may be various general and / or special processing components with processing and computing capabilities. For example, some examples of computing units may include a central processing unit, a graphics processing unit, various special artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc.

[0034] In the embodiment of the present disclosure, the target virtual image can be various virtual images, for example, the target virtual image can be various images such as a virtual person, a virtual animal, etc.

[0035] In operation S220, in response to obtaining the target image, the target image is encoded by using a target computing unit to obtain target video sequence data.

[0036] In the embodiment of the present disclosure, the target computing unit may be used to perform rendering and encoding operations. For example, a graphics processing unit may be used to perform the above rendering operations. Next, the graphics processing unit may also be used to perform encoding operations.

[0037] In the disclosed embodiment, the encoding operation may be a video encoding operation. For example, the target image may be encoded based on various video encoding standards. In one example, the video encoding standard may be the 4th generation Moving Picture Experts Group 4 (MP4) encoding standard.

[0038] In operation S230, current video data is obtained according to the previous video sequence data at the target moment and the target video sequence data.

[0039] In the embodiment of the present disclosure, the previous video sequence data of the target moment corresponds to the previous time period of the target moment, and the previous time period of the target moment includes at least one moment before the target moment. For example, the previous time period may include all moments before the target moment.

[0040] In the disclosed embodiment, the target video sequence data may be merged with the previous video sequence data at the target moment to obtain the current video data.

[0041] In operation S240, in response to determining that the target time satisfies a preset condition, a target video is obtained according to the current video data.

[0042] In the embodiment of the present disclosure, the preset condition may be various conditions. For example, the preset condition may include: the target moment is a preset moment among multiple moments. The preset moment may be any moment among the multiple moments (for example, the second to last moment or the last moment).

[0043] Through the embodiments of the present disclosure, by using the target computing unit to perform rendering and video encoding operations, the time used for data transmission between the computing unit (such as a graphics processing unit) and the storage unit can be saved, and the time used by other computing units (such as a central processing unit) to read data from the storage unit can also be saved.

[0044] In addition, through the disclosed embodiment, the target computing unit is used to perform rendering and video encoding operations, which also saves the hard disk space required for storing multiple images corresponding to multiple action driving coefficients. In addition, after obtaining the target image, the target computing unit can be used to perform video encoding. After the encoding of the target image is completed, the target image can be deleted from the cache, which can effectively save the cache space occupied by the target image.

[0045] It can be understood that the above describes the method flow of the present disclosure, and the following will describe multiple action driving coefficients of the present disclosure.

[0046] In some embodiments, the plurality of action driving coefficients may be K action driving coefficients. The K action driving coefficients correspond to K moments respectively, the target action driving coefficient is the i-th action driving coefficient, the target moment is the i-th moment, i is an integer greater than 1 and less than or equal to K, and K is an integer greater than 1.

[0047] In the embodiment of the present disclosure, the plurality of action driving coefficients are obtained by processing target data using a preset algorithm, and the target data includes at least one of a target text and a target audio. For example, the target data may be a target text. Among the plurality of action driving coefficients, at least one action driving coefficient may correspond to a character of the target text.

[0048] In the disclosed embodiment, the preset algorithm may include a speech animation synthesis algorithm. For example, after being processed by the preset algorithm, the timestamp of each character in the target text may be obtained. According to the timestamp of each character, at least one moment of at least one action driving coefficient corresponding to the character may be determined.

[0049] In the disclosed embodiment, the action driving coefficient may be the above-mentioned mixed shape coefficient. The action driving coefficient may be applied to multiple mixed shape bases of the face of the virtual image, so that the shapes of these mixed shape bases change so that the virtual image performs corresponding expression actions. Through the disclosed embodiment, the action driving coefficient is used to drive the target virtual image to perform expression actions, so that the target virtual image can show the corresponding lip shape when broadcasting content, which improves the authenticity of the target virtual image and helps to improve the user experience.

[0050] It can be understood that the above describes the action driving coefficient of the present disclosure, and the preset conditions will be described below in conjunction with relevant embodiments.

[0051] In some embodiments, the preset condition may include: the target moment is the last moment among multiple moments. For example, in the case of i=K, it can be determined that the i-th moment as the target moment satisfies the preset condition. For another example, in the case of i<K, it can be determined that the i-th moment as the target moment does not satisfy the preset condition. Through the embodiments of the present disclosure, multiple action drive coefficients can be rendered as images, and then a video corresponding to the target text or target audio can be generated, and a virtual image video can be generated quickly and conveniently, which improves the user experience.

[0052] It can be understood that the action driving coefficient and the preset conditions of the present disclosure are described above, and the method of the present disclosure will be further described below in conjunction with relevant embodiments.

[0053] Figure 3 is a flowchart of a video generation method according to an embodiment of the present disclosure.

[0054] like Figure 3 As shown, method 300 may include operations S310 to S330 and operations S341 to S343. In addition, in the embodiment of the present disclosure, Figure 3 Before the operation S310 shown, rendering using the target computing unit may include: performing rendering using the target computing unit according to the first action driving coefficient to obtain the first image. According to the first image, obtaining the previous video sequence data at the i-th moment.

[0055] For example, after receiving K action drive coefficients, the first action drive coefficient may be used as the target action drive coefficient. The graphics processing unit may render the target virtual image for the first time to obtain the first image. Next, the graphics processing unit may also perform video encoding on the first image to obtain the first video sequence data as the previous video data at the i-th moment. It can be understood that i is an integer greater than 1 and less than or equal to K. Next, operation S310 may be performed.

[0056] In operation S310, a target virtual image is rendered using a target computing unit according to an i-th action driving coefficient among K action driving coefficients to obtain an i-th image.

[0057] In the embodiment of the present disclosure, in the process of rendering the target virtual image, the target virtual image may be rendered based on a preset material. For example, the preset material may include a preset background material. For another example, the graphics processing unit may be deployed with various rendering engines. According to the i-th action driving coefficient, rendering is performed to obtain the i-th rendering result. The i-th rendering result is merged with the preset background material to obtain the i-th image.

[0058] In operation S320, in response to obtaining the i-th image, the i-th image is encoded using a target computing unit to obtain i-th video sequence data.

[0059] In the embodiment of the present disclosure, after the i-th image is obtained, the i-th image may be video encoded using a graphics processing unit. For example, the graphics processing unit may be deployed with a Fast Forward MPEG (FFmpeg) video synthesis engine. Thus, the graphics processing unit may be used to perform video encoding.

[0060] In operation S330, current video data is obtained according to the previous video sequence data at the ith moment and the ith video sequence data.

[0061] For example, the i-th video sequence data may be merged with the previous video sequence data at the i-th moment to obtain the current video data.

[0062] Next, it can be determined whether the i-th moment satisfies a preset condition. In the embodiment of the present disclosure, the preset condition may include: the i-th moment is the last moment among K moments. For example, it can be determined whether i is equal to K.

[0063] In operation S341, it is determined whether i is equal to K.

[0064] In the embodiment of the present disclosure, in response to determining that i is not equal to K, it can be determined that the target time does not meet the preset condition, and operation S342 can be performed.

[0065] In the embodiment of the present disclosure, in response to determining that i is equal to K, it can be determined that the target time satisfies a preset condition, and operation S343 can be performed.

[0066] In operation S342, the current video data is determined as the previous video sequence data at the i+1th moment.

[0067] In the embodiment of the present disclosure, when i is not equal to K, the current video data obtained from the i-th video sequence data can be used as the previous video sequence data at the i+1-th moment. Next, according to other action drive coefficients, at least one operation S310 to operation S330 can be performed. Until operations S310 to operation S330 are performed according to the K-th action drive coefficient. Through the embodiment of the present disclosure, images can be continuously rendered and video encoding can be performed when the preset conditions are not met at the target moment. In this process, the image is encoded into video sequence data after it is generated, and the video sequence data can be fused with the previous video sequence data to obtain the current video data. Therefore, in the rendering and video encoding process, the storage space required for the image is greatly reduced, which saves storage resources and also helps to improve the efficiency of video generation.

[0068] In operation S343, a target video is obtained according to the current video data.

[0069] For example, after fusing the K-th video training data with the previous video sequence data at the K-th moment, the current video data obtained can be used as the target video.

[0070] Through the embodiments of the present disclosure, the graphics processing unit has a strong image processing capability, and the efficiency of video generation can be improved by using the graphics processing unit for rendering and encoding.

[0071] It can be understood that the above target calculation unit can be deployed on the server, and the target text used to determine the action driving coefficient can be obtained from the client. The method of the present disclosure will be further described below in conjunction with the client and the server.

[0072] Figure 4 is a flowchart of a video generating method according to another embodiment of the present disclosure.

[0073] like Figure 4 As shown, the method 400 can be performed by the client and the server together. The client can perform operations S4101 to S4105.

[0074] In operation S4101, a target text input by a user is acquired.

[0075] For example, the user may input the target text on the interactive interface of the client. The target text Text2 input by the user may be "Hello, welcome to experience the speech animation synthesis technology. Next, let's take a look at a piece of news: Product A released by XXX is officially open for targeted internal testing."

[0076] In operation S4102, request data for generating a video is constructed and sent.

[0077] For example, the target text TextT2 and related information may be used as request data. The client may send the request data to the server.

[0078] Next, the server may receive the request data and perform operation 4201 according to the request data.

[0079] In operation S4201, a preset material is acquired.

[0080] For example, the above-mentioned preset background material can be obtained from a preset material library.

[0081] In operation 4202, target audio is obtained.

[0082] For example, the target text Text2 may be converted into a target audio based on a text-to-speech (TTS) algorithm.

[0083] In operation S4203, the target text is processed using a preset algorithm to obtain a processing result.

[0084] For example, the target text may be processed using a speech animation synthesis algorithm to obtain a processing result. For another example, the processing result may include multiple moments related to the target text and multiple action drive coefficients corresponding to the target text. It is understood that operation S4203 may be performed by a central processing unit of the server.

[0085] In operation S4200, the target virtual image is rendered to obtain a target image; and encoding is performed based on the target image to obtain a target video.

[0086] It is understood that operation S4200 may be performed by a graphics processing unit of the server. It is also understood that operation S4200 is the same or similar to the above-mentioned method 200 or method 300, and the present disclosure will not repeat them here.

[0087] It can be understood that the video format of the target video can be the 4th generation dynamic picture expert group format, the video format with transparent channel (such as mov) and the webm format. The video in the 4th generation dynamic picture expert group format can be obtained based on h264 encoding. The video with transparent channel can be obtained based on qtrle encoding. The webm format can be obtained based on vp9 encoding.

[0088] In operation S4204, a target video is uploaded and a target video link is generated.

[0089] For example, the target video can be uploaded to a cloud storage. For another example, a link to the target video can also be generated. It is understood that the link can be a Uniform Resource Locator (URL).

[0090] In operation S4205, the video generation status is updated.

[0091] For example, the video generation status may be updated to "complete".

[0092] It can be understood that after the client executes operation S4102, the client may also execute operation S4103.

[0093] In operation S4103, it is confirmed that the request is successful.

[0094] For example, the client can obtain relevant information from the server to confirm whether the request is successful.

[0095] In operation S4104, the generation status is polled.

[0096] For example, the video generation status may be queried at preset time intervals.

[0097] It can be understood that after operation S4205, when the client queries the video generation status next time, the video generation status that can be queried is “completed.” In addition, after operation S4205, the server can perform operation S4206 in response to receiving the query request from the client.

[0098] In operation S4206, the target video link is sent.

[0099] For example, the target video link generated in operation S4204 may be sent to the client.

[0100] After receiving the target video link, the client may perform operation S4105.

[0101] In operation S4105, the target video link is returned.

[0102] For example, the client may display the target video link on a related interactive interface to return the link to the user.

[0103] The method of the present disclosure will be described below with reference to relevant schematic diagrams.

[0104] Figure 5 is a schematic diagram of an interactive interface according to an embodiment of the present disclosure.

[0105] The user can enter the target text in the interactive interface. For example, the target text Text2 can be "Hello, welcome to experience the speech animation synthesis technology. Next, let's take a look at a piece of news: Product A released by XXX is officially open for targeted internal testing." Figure 5 shown.

[0106] In the disclosed embodiment, the preset news material can also be merged with the rendering result. For example, the preset news material M520 can be merged with a rendering result to obtain an image. It can be understood that the preset news material M520 corresponds to the "news" mentioned in the target text T510.

[0107] It is understandable that the client can send the target text T510 of the interactive interface to the server so that the server receives the target text. After receiving the target text, the server can generate a video.

[0108] Figure 6 is a schematic diagram of a video frame according to an embodiment of the present disclosure.

[0109] like Figure 6 As shown, Figure 6 As shown, the video frame F600 displays a preset news material M620 and a target virtual image 630 .

[0110] It can be understood that the above description uses Chinese characters as an example. However, the present disclosure is not limited to this. In the embodiments of the present disclosure, the characters may also be characters in other languages. For example, the characters may be English characters. For another example, the target text Text3 may be "Hello, welcome to experience voice animation synthesistechnology". The above-mentioned preset algorithm may also process the target text Text3 to obtain multiple action driving coefficients of the target text Text3. Next, according to the above-mentioned method 200, video generation may be performed.

[0111] Figure 7 is a block diagram of a video generating apparatus according to an embodiment of the present disclosure.

[0112] like Figure 7 As shown, the apparatus 700 may include a rendering module 710 , an encoding module 720 , a first obtaining module 730 , and a second obtaining module 740 .

[0113] The rendering module 710 is used to render the target virtual image using the target calculation unit according to the target action driving coefficient among the multiple action driving coefficients to obtain the target image. For example, the multiple action driving coefficients correspond to multiple moments respectively, and the target action driving coefficient corresponds to the target moment among the multiple moments.

[0114] The encoding module 720 is used for encoding the target image using the target computing unit in response to obtaining the target image to obtain the target video sequence data.

[0115] The first acquisition module 730 is used to obtain the current video data according to the previous video sequence data at the target moment and the target video sequence data. For example, the previous video sequence data at the target moment corresponds to the previous time period of the target moment, and the previous time period of the target moment includes at least one moment before the target moment.

[0116] The second obtaining module 740 is configured to obtain a target video according to current video data in response to determining that the target moment satisfies a preset condition.

[0117] In some embodiments, the preset condition includes: the target moment is the last moment among the multiple moments.

[0118] In some embodiments, there are K multiple action driving coefficients, the K action driving coefficients correspond to K moments respectively, the target action driving coefficient is the i-th action driving coefficient, the target moment is the i-th moment, i is an integer greater than 1 and less than K, K is an integer greater than 1, and the device 700 also includes: a determination module, which is used to determine the current video data as the previous video sequence data of the i+1-th moment in response to determining that the target moment does not meet the preset conditions.

[0119] In some embodiments, the rendering module includes: a rendering unit, used to render using a target computing unit according to a first action driving coefficient to obtain a first image; and an acquisition unit, used to obtain the previous video sequence data at the i-th moment according to the first image.

[0120] In some embodiments, the plurality of action driving coefficients are obtained by processing target data using a preset algorithm, where the target data includes at least one of a target text and a target audio.

[0121] In some embodiments, the target data is a target text, and characters of the target text correspond to at least one action driving coefficient.

[0122] In some embodiments, the target computing unit comprises a graphics processing unit.

[0123] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0124] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0125] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0126] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0127] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0128] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the video generation method. For example, in some embodiments, the video generation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the video generation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the video generation method in any other appropriate manner (e.g., by means of firmware).

[0129] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0131] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) display or an LCD (liquid crystal display)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0133] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0134] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0135] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0136] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A video generation method, comprising: Rendering a target virtual image using a graphics processing unit according to a target action driving coefficient among the plurality of action driving coefficients to obtain a target image, wherein the target image is stored in a cache, the plurality of action driving coefficients correspond to a plurality of moments respectively, and the target action driving coefficient corresponds to a target moment among the plurality of moments; In response to obtaining the target image, using the graphics processing unit to perform video encoding on the target image to obtain target video sequence data, and deleting the target image from the cache; Merging the previous video sequence data of the target moment with the target video sequence data to obtain current video data, wherein the previous video sequence data of the target moment corresponds to a previous time period of the target moment, and the previous time period of the target moment includes at least one of the moments before the target moment; and In response to determining that the target moment satisfies a preset condition, a target video is obtained according to the current video data, wherein the preset condition includes: the target moment is the last moment among the multiple moments.

2. The method according to claim 1, wherein: There are K multiple action driving coefficients, the K action driving coefficients correspond to K moments respectively, the target action driving coefficient is the i-th action driving coefficient, the target moment is the i-th moment, i is an integer greater than 1 and less than K, and K is an integer greater than 1. The method further comprises: In response to determining that the target moment does not satisfy the preset condition, the current video data is determined to be the previous video sequence data of the (i+1)th moment.

3. The method according to claim 1, wherein: The rendering of the target virtual image by using the graphics processing unit comprises: Rendering using the graphics processing unit according to the first motion driving coefficient to obtain a first image; and According to the first image, the previous video sequence data at the i-th moment is obtained.

4. The method according to claim 1, wherein: The plurality of action driving coefficients are obtained by processing target data using a preset algorithm, and the target data includes at least one of a target text and a target audio.

5. The method according to claim 4, wherein: The target data is a target text, and characters of the target text correspond to at least one of the action driving coefficients.

6. A video generating device, comprising: a rendering module, configured to render a target virtual image using a graphics processing unit according to a target action driving coefficient among the plurality of action driving coefficients to obtain a target image, wherein the target image is stored in a cache, the plurality of action driving coefficients respectively correspond to a plurality of moments, and the target action driving coefficient corresponds to a target moment among the plurality of moments; an encoding module, configured to, in response to obtaining the target image, perform video encoding on the target image using the graphics processing unit to obtain target video sequence data, and delete the target image from the cache; a first obtaining module, configured to fuse the previous video sequence data of the target moment with the target video sequence data to obtain current video data, wherein the previous video sequence data of the target moment corresponds to a previous time period of the target moment, and the previous time period of the target moment includes at least one of the moments before the target moment; and The second obtaining module is used to obtain the target video according to the current video data in response to determining that the target moment meets a preset condition, wherein the preset condition includes: the target moment is the last moment among the multiple moments.

7. The device according to claim 6, wherein: There are K multiple action driving coefficients, the K action driving coefficients correspond to K moments respectively, the target action driving coefficient is the i-th action driving coefficient, the target moment is the i-th moment, i is an integer greater than 1 and less than K, and K is an integer greater than 1. The device also includes: The determination module is used for determining the current video data as the previous video sequence data of the (i+1)th moment in response to determining that the target moment does not satisfy the preset condition.

8. The device according to claim 6, wherein: The rendering module includes: a rendering unit, configured to perform rendering using the graphics processing unit according to the first motion driving coefficient to obtain a first image; and The obtaining unit is used to obtain the previous video sequence data at the i-th moment according to the first image.

9. The device according to claim 6, wherein: The plurality of action driving coefficients are obtained by processing target data using a preset algorithm, and the target data includes at least one of a target text and a target audio.

10. The device according to claim 9, wherein: The target data is a target text, and characters of the target text correspond to at least one of the action driving coefficients.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

13. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images