Method and device for generating video data, storage medium, and electronic device

Abstract

The present invention provides a method and device for generating video data, a storage medium, and an electronic device, wherein the method comprises: loading a first video on a first client, and detecting an editing request for the first video; responding to the editing request, obtaining source video template data of the first video, wherein the source video template data comprises a virtual character model that allows editing; obtaining character feature data of the virtual character model; and rendering and generating a second video in a virtual scene model template corresponding to the source video template data based on the character feature data. The present invention solves the technical problem that the related art can only edit videos through image special effects, and realizes deep editing of videos.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for generating video data, a storage medium, and an electronic device. Background Art

[0002] In related technologies, videos are shot by users or edited based on existing videos. If users want to play a role in a certain scene, on the one hand, they can only rebuild the scene to shoot, which is costly and the effect is not as expected. On the other hand, it can be achieved by adding some image special effects in the later stage. The edited visual effect is effective, but deep editing is not allowed.

[0003] With respect to the above-mentioned problems existing in the related technologies, no effective solutions have been found so far. Summary of the invention

[0004] Embodiments of the present invention provide a method and device for generating video data, a storage medium, and an electronic device.

[0005] According to one embodiment of the present invention, a method for generating video data is provided, comprising: loading a first video on a first client and detecting an editing request for the first video; in response to the editing request, obtaining source video template data of the first video, wherein the source video template data includes a virtual character model that allows editing; obtaining character feature data of the virtual character model; and based on the character feature data, rendering and generating a second video in a virtual scene model template corresponding to the source video template data.

[0006] According to another embodiment of the present invention, there is provided a device for generating video data, comprising: a detection module, used to load a first video on a first client, and detect an editing request for the first video; a first acquisition module, used to respond to the editing request, and obtain source video template data of the first video, wherein the source video template data includes a virtual character model that allows editing; a second acquisition module, used to obtain character feature data of the virtual character model; and a generation module, used to render and generate a second video in a virtual scene model template corresponding to the source video template data based on the character feature data.

[0007] According to yet another embodiment of the present invention, a storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when running.

[0008] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0009] Through the present invention, a first video is loaded on a first client, and an editing request for the first video is detected. In response to the editing request, source video template data of the first video is obtained, wherein the source video template data includes a virtual character model that allows editing, and character feature data of the virtual character model is obtained; based on the character feature data, a second video is rendered and generated in a virtual scene model template corresponding to the source video template data, and by obtaining the source video template data of the first video and the character feature data of the virtual character model therein, the character feature data can be used to configure or control the virtual character model in the virtual scene model template, and the first video can be re-edited, thereby solving the technical problem that the related technology can only edit videos through image special effects, and realizing deep editing of videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0011] Figure 1 It is a hardware structure block diagram of a video data generation server according to an embodiment of the present invention;

[0012] Figure 2 is a flow chart of a method for generating video data according to an embodiment of the present invention;

[0013] Figure 3 is a schematic diagram of a hierarchical tree according to an embodiment of the present invention;

[0014] Figure 4 is a schematic diagram of video editing according to an embodiment of the present invention;

[0015] Figure 5 is a structural block diagram of a device for generating video data according to an embodiment of the present invention;

[0016] Figure 6 It is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] Example 1

[0020] The method embodiment provided in the first embodiment of the present application can be executed in a game server, a cloud server, a computer or a similar electronic terminal. Taking running on a server as an example, Figure 1 FIG. 1 is a hardware structure diagram of a video data generation server according to an embodiment of the present invention. Figure 1 As shown, the server may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the server may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0021] The memory 104 can be used to store server programs, for example, software programs and modules of application software, such as a server program corresponding to a method for generating video data in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the server program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. In this embodiment, the processor 104 is used to respond to human-computer interaction instructions and game strategies, and control the target virtual character to perform specified operations to complete the game task. The memory 104 is used to store program scripts, configuration information, attribute information of the virtual character, etc.

[0022] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0023] Optionally, the input-output device 108 further includes a human-computer interaction screen for obtaining human-computer interaction instructions through a human-computer interaction interface and for presenting streaming media images;

[0024] In this embodiment, a method for generating video data is provided. Figure 2 FIG. 1 is a flow chart of a method for generating video data according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0025] Step S202, loading a first video on a first client, and detecting an editing request for the first video;

[0026] Step S204, in response to the editing request, obtaining source video template data of the first video, wherein the source video template data includes a virtual character model that allows editing;

[0027] The video server pre-configures multiple virtual videos, such as movie clips, game videos, etc. The first client selects the first video and loads it for playback. The parameters of specific character elements in the first video are set to be replaceable, while others are not replaceable (such as scenery, props, non-key characters, etc.). When the user edits and generates a new second video, he or she selects at least one favorite character in the first video as an object for replacement or processing. Optionally, the source video template data also includes virtual character models or scene models (such as buildings, mountains, NPCs, etc.) that are not allowed to be edited, and the configuration data of such models is fixed.

[0028] Step S206, obtaining character feature data of the virtual character model;

[0029] Optionally, the character feature data may be three-dimensional facial feature data, action feature data, body feature data, or sound feature data.

[0030] Step S208, rendering and generating a second video in a virtual scene model template corresponding to the source video template data according to the character feature data;

[0031] Optionally, the character feature data is used to configure or control the virtual character model in the virtual scene model template to achieve depth rendering.

[0032] Optionally, the first video is loaded on a video playback terminal (such as a live broadcast terminal, a television, a video conferencing terminal, a game terminal, an instant messaging terminal), and a virtual picture is presented in real time, wherein the virtual picture includes a virtual character rendered based on a virtual character model.

[0033] Through the above steps, the first video is loaded on the first client, and the editing request of the first video is detected, and the editing request is responded to, and the source video template data of the first video is obtained, wherein the source video template data includes a virtual character model that allows editing, and the character feature data of the virtual character model is obtained; according to the character feature data, the second video is rendered and generated in the virtual scene model template corresponding to the source video template data, and by obtaining the source video template data of the first video and the character feature data of the virtual character model therein, the character feature data can be used to configure or control the virtual character model in the virtual scene model template, and the first video is re-edited, which solves the technical problem that the related technology can only edit videos through image special effects, and realizes deep editing of videos.

[0034] In one implementation of this embodiment, loading the first video on the first client includes: generating a game clip video using screen frames of the virtual game during loading of the virtual game on the first client; and loading the game clip video.

[0035] The present embodiment can obtain character feature data from multiple data sources, and obtaining the character feature data of the virtual character model includes: capturing at least one of the following of the virtual character model from the real scene data collected by the first client: three-dimensional facial feature data, action feature data, body feature data, and sound feature data; and / or, requesting at least one of the following of the virtual character model from the feature material library: three-dimensional facial feature data, action feature data, body feature data, and sound feature data; selecting a game character in a virtual game, and reading the character feature data of the virtual character model from the character attributes or skill attributes of the game character.

[0036] Optionally, the three-dimensional facial feature data may be three-dimensional data of facial features, three-dimensional data of specific facial organs, etc., the motion feature data may be continuous dynamic movements of a real user, such as martial arts movements, dance movements, drama movements, etc., the body feature data may be data such as the height, weight, body proportions, etc. of the real user, and the sound feature data may be feature data such as timbre and voiceprint.

[0037] In one example, when collecting character feature data, the depth camera of the user's mobile phone is used to collect feature information of the user's face and other parts in real time, and the collected facial feature information is used to control the virtual character model to present corresponding expressions, thereby generating a second video that can reflect the user's real-time expression during the video shooting process. On the other hand, character feature data can also be obtained from existing real-life images. During the collection process, a panoramic scan of the user's face or body is required to generate a skeleton model of the virtual character model. In addition, the prototype of the virtual character in the game can also be obtained, and the virtual image can be further shaped based on the virtual character prototype.

[0038] Character feature data can be collected in real time through the client, or requested from the feature material library. The feature material library pre-stores multiple sets of character feature data, and each set of character feature data can be adapted to different types of virtual character models. Since the character attributes of the game character store the character image, voice and other feature data of the game character, the character feature data can be read from the character attributes, and the skill attributes store the special effects animation or action features when the corresponding skill is triggered, so the character feature data can also be read from the skill attributes. When requesting character feature data from the feature material library, the above data is purchased by consuming a certain amount of virtual acquisition, which can be the input data of action stars, well-known bloggers, etc., the voice data of singers, etc. can be used as character feature data.

[0039] Optionally, the virtual character model may be a player-controlled character (PCC), a non-player-controlled character (NPC), etc., and obtaining the character feature data of the virtual character model may be, but is not limited to: obtaining the first character feature data of the PCC model from the first client; obtaining the second character feature data of the NPC model from the first client; obtaining the third character feature data of the first virtual character model from the first client, and obtaining the fourth character feature data of the second virtual character model from the second client.

[0040] When acquiring character feature data of a virtual character model, this embodiment may acquire the data from multiple clients. For example, character feature data of the same virtual character model may be acquired from multiple clients, such as one client acquiring voice features of user A and one client acquiring action features of user B to achieve a combined visual effect. Character feature data may also be acquired separately for virtual character models, such as acquiring third character feature data of a first virtual character model from a first client and acquiring fourth character feature data of a second virtual character model from a second client. Different virtual character models may be used to link the first client and the second client to control the virtual character models in the video respectively. The linkage may be synchronous or asynchronous.

[0041] In one implementation of this embodiment, the physics engine of the game server is called to generate the second video:

[0042] In one implementation scenario, the character feature data includes third character feature data of the first virtual character model and fourth character feature data of the second virtual character model. Based on the character feature data, rendering and generating the second video in the virtual scene model template corresponding to the source video template data includes:

[0043] S11, calling the game scene sub-model, world motion sub-model, first virtual character sub-model, second virtual character sub-model, and game interaction sub-model of the virtual scene model template from the game rendering engine;

[0044] Among them, the game scene sub-model includes models of dynamic and static scenes. The world motion sub-model is the motion logic model of interactive objects. It depends on the type of game plot. After the game server develops the interface, it can be called externally. In the virtual scene of the Metaverse, the game development company will develop this editing interface and render and output the second video in the cloud server of the Metaverse community.

[0045] S12, using the third character feature data to edit the first virtual character sub-model in the game scene sub-model and the world motion sub-model, and using the fourth character feature data to edit the second virtual character sub-model in the game scene sub-model and the world motion sub-model, to generate an intermediate video;

[0046] S13, using the game interaction sub-model to add interaction information between the first virtual character sub-model and the second virtual character sub-model in the intermediate video to generate a second video.

[0047] In one example, when the game interaction sub-model is used to add interaction information between the first virtual character sub-model and the second virtual character sub-model in the intermediate video, interaction features such as body movements, language social interaction (such as real-time social conversation between two players, acting out a video plot together) are extracted from the third character feature data of the first virtual character sub-model and the fourth character feature data of the second virtual character model, that is, the social interaction between the first virtual character sub-model and the second virtual character sub-model.

[0048] In some scenarios of this embodiment, the first video can be obtained from the video social software and edited based on the first video, so as to connect the video social software and the game server, use the game server as the underlying technical support, and make full use of the distribution capabilities and popularity traffic of the video social software.

[0049] Based on the above scenario, the first client is a video social client, and the first video is loaded on the first client, and an editing request for the first video is detected, including: loading the first video on the video social client, and detecting the editing request for the first video in a playback interface of the first video played on the video social client, wherein the editing request is carried in video frame data of the first video.

[0050] Among them, the code of the editing request carries the interface script of the game server. By triggering, the editing resources of the game server can be called. After the editing is completed and the second video is generated, it can continue to be published on the video social software.

[0051] In one implementation of this embodiment, after the first client loads the first video, the method further includes:

[0052] S21, extracting an editor tag from the first video, wherein the editor tag is used to identify an editing account related to the first video;

[0053] S22, calculating traffic data generated by the first video on the first client;

[0054] S23, saving the traffic data to the billing account of the editing account.

[0055] Optionally, saving the traffic data to the billing account of the editing account includes: extracting the parent account and child account of the first video from the hierarchical tree, wherein the parent account is the editing account of the source video of the first video, and the child account is the editing account of the first video; calculating the editing ratios of the parent account and the child account to the first video respectively; dividing the traffic data according to the editing ratios, and saving the divided traffic sub-data to the parent account and the child account respectively.

[0056] Figure 3 It is a schematic diagram of the hierarchical tree of an embodiment of the present invention, illustrating three levels, wherein the first level includes account 1, the second level includes account 2 and account 3, and the third level includes account 4, account 5, account 6, and account 7. Taking account 5 as an example, the child account for editing the first video, its parent account includes account 2 and account 1. Account 1 first edits video A based on the original video, account 2 edits and generates video B based on video A, and account 5 edits and generates the first video based on video B. If the editing ratio of account 1 to the first video is 10%, the editing ratio of account 2 to the first video is 30%, and the editing ratio of account 5 to the first video is 60%, when the traffic data of the first video is divided, account 1, account 2 and account 5 each get 10%, 30% and 60% respectively.

[0057] In this embodiment, the link of the editing request is carried in the video frame data of the first video, and is displayed on the playback interface of the first video by recording it so that other viewers can find it and initiate such a video editing request. The platform records the video editing and distributes it to each editor in proportion to their contribution when calculating the traffic share, thereby promoting video editing and distribution, and promoting distribution and re-creativity on the platform.

[0058] In another implementation of this embodiment, the image processing engine of the game server is called to generate the second video:

[0059] In an implementation scenario of this embodiment, based on the character feature data, rendering and generating the second video in the virtual scene model template corresponding to the source video template data includes:

[0060] S31, deleting the source virtual character corresponding to the virtual character model in the virtual scene model template to obtain an intermediate virtual scene model template;

[0061] S32, playing the intermediate virtual scene model template from the starting position, and adding the virtual character model to the playing screen of the intermediate virtual scene model template;

[0062] The source virtual character and the intermediate virtual scene model template of this embodiment respectively correspond to a part of the source video template data. After deleting the data of the source virtual character from the source video template data, the intermediate virtual scene model template is loaded and played, and the virtual character model is re-added. The virtual character model is a configurable original model.

[0063] S33, controlling the virtual character model to move in the playback picture to form a motion track based on the character feature data, and recording the motion track;

[0064] In some examples, controlling the movement of a virtual character model in a playback screen to form an action trajectory based on character feature data includes: generating control instructions for the virtual character model in the playback screen based on the character feature data, and adding multiple virtual cameras corresponding to multiple perspectives of the playback screen, wherein the control instructions are used to instruct the virtual character model to interact with scene elements in the playback screen; in response to the control instructions, controlling the virtual character model to move in the playback screen to form a first motion trajectory, and controlling the multiple virtual cameras to capture multiple groups of dynamic images of the virtual character model during the movement along the first motion trajectory.

[0065] In some examples, controlling the movement of a virtual character model in a playback screen based on character feature data to form a motion trajectory includes: extracting facial features and body posture features in the character feature data in sequence according to a storage sequence, wherein the character feature data stores a number of groups of facial features and a number of groups of body posture features in a queue based on the storage sequence; generating facial movement instructions and body movement instructions corresponding to the facial features and body posture features, respectively, and using the facial movement instructions and the body movement instructions to synchronously control the movement of the target virtual image in the playback screen to form a second motion trajectory.

[0066] S34, re-rendering the virtual character model in the intermediate virtual scene model template along the action trajectory to generate a second video containing the target virtual character.

[0067] In some examples, re-rendering the virtual character model in the intermediate virtual scene model template along the action trajectory to generate a second video containing the target virtual character includes: obtaining multiple groups of dynamic images from multiple virtual cameras; for each frame of the playback image of the intermediate virtual scene model template, selecting a group of video images from the multiple groups of dynamic images, and editing the playback images and video images frame by frame to generate a second video containing the target virtual character.

[0068] The user can enter the virtual scene model template corresponding to the first video through the virtual character of his / her choice. In the process of controlling the virtual character, the user can perform the plot corresponding to the first video with the virtual characters or NPCs controlled by other users, or interact with other virtual characters and NPCs on his / her own. The performance process is filmed by multiple virtual cameras in the virtual scene, and later the images captured by multiple virtual cameras can be edited to obtain video clips of the second video.

[0069] Figure 4 It is a schematic diagram of video editing according to an embodiment of the present invention. The first video screenshot and the second video screenshot are screenshots of the same video position in the first video and the second video respectively. Character 1 and character 2 are configured and rendered based on the same virtual character model. By reconfiguring the virtual character model using character feature data, the images in other areas remain unchanged, thereby achieving video editing.

[0070] In one implementation of the present embodiment, obtaining character feature data of the virtual character model includes: collecting a first target audio, extracting timbre features from the first target audio, and extracting dialogue information of the virtual character model in the first video; and generating character dubbing data of the virtual character model based on the timbre features and dialogue information.

[0071] By adopting this implementation mode, the virtual character model in the first video can be re-dubbed while keeping the dialogue information unchanged. For example, the timbre of a character in the first video can be edited into the user's own timbre while the video picture remains unchanged.

[0072] In another implementation of the present embodiment, obtaining character feature data of the virtual character model includes: collecting a second target audio, parsing text information in the second target audio; configuring the second target audio and the text information as character dubbing data and character dialogue data of the virtual character model, respectively; identifying lip features and body movement features in the second target audio, and configuring the lip features and body movement features as lip movement data and body movement data of the virtual character model, respectively.

[0073] By adopting this implementation mode, the dubbing, lines, actions, etc. of the virtual character model in the first video can be edited.

[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0075] Example 2

[0076] In the present embodiment, a device for generating video data is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0077] Figure 5 is a structural block diagram of a device for generating video data according to an embodiment of the present invention. Figure 5 As shown, the device includes: a detection module 50, a first acquisition module 52, a second acquisition module 54, and a generation module 56, wherein:

[0078] A detection module 50, configured to load a first video on a first client and detect an editing request for the first video;

[0079] A first acquisition module 52, configured to respond to the editing request and acquire source video template data of the first video, wherein the source video template data includes a virtual character model that allows editing;

[0080] A second acquisition module 54 is used to acquire character feature data of the virtual character model;

[0081] The generating module 56 is used to render and generate a second video in the virtual scene model template corresponding to the source video template data according to the character feature data.

[0082] Optionally, the detection module includes: a generation unit, used to generate a game clip video using screen frames of the virtual game during the process of loading the virtual game on the first client; and a loading unit, used to load the game clip video.

[0083] Optionally, the second acquisition module includes at least one of the following: a first acquisition unit, used to acquire first character feature data of the player-controlled character PCC model from the first client; a second acquisition unit, used to acquire second character feature data of the non-player character NPC model from the first client; a third acquisition unit, used to acquire third character feature data of the first virtual character model from the first client, and acquire fourth character feature data of the second virtual character model from the second client.

[0084] Optionally, the character feature data includes third character feature data of the first virtual character model and fourth character feature data of the second virtual character model, and the generation module includes: a calling unit, used to call the game scene sub-model, world motion sub-model, first virtual character sub-model, second virtual character sub-model, and game interaction sub-model of the virtual scene model template from the game rendering engine; an editing unit, used to edit the first virtual character sub-model in the game scene sub-model and the world motion sub-model using the third character feature data, and to edit the second virtual character sub-model in the game scene sub-model and the world motion sub-model using the fourth character feature data to generate an intermediate video; a first generation unit, used to add interaction information between the first virtual character sub-model and the second virtual character sub-model in the intermediate video using the game interaction sub-model to generate the second video.

[0085] Optionally, the first client is a video social client, and the detection module includes: a detection unit, used to load the first video in the video social client, and detect an editing request for the first video in a playback interface of the video social client playing the first video, wherein the editing request is carried in video frame data of the first video.

[0086] Optionally, the device also includes: an extraction module, used to extract an editor tag from the first video after the detection module loads the first video on the first client, wherein the editor tag is used to identify an editing account related to the first video; a calculation module, used to calculate the traffic data generated by the first video on the first client; and a saving module, used to save the traffic data to a billing account of the editing account.

[0087] Optionally, the saving module includes: an extraction unit, used to extract the parent account and child account of the first video from the hierarchical tree, wherein the parent account is the editing account of the source video of the first video, and the child account is the editing account of the first video; a calculation unit, used to calculate the editing ratios of the parent account and the child account to the first video respectively; a saving unit, used to divide the traffic data according to the editing ratio, and save the divided traffic sub-data to the parent account and the child account respectively.

[0088] Optionally, the second acquisition module includes: a fourth acquisition unit, used to capture at least one of the following of the virtual character model from the real-scene data collected by the first client: three-dimensional facial feature data, action feature data, body feature data, and sound feature data; and / or a fifth acquisition unit, used to request at least one of the following of the virtual character model from a feature material library: three-dimensional facial feature data, action feature data, body feature data, and sound feature data; a sixth acquisition unit, used to select a game character in a virtual game and read the character feature data of the virtual character model from the character attributes or skill attributes of the game character.

[0089] Optionally, the generation module includes: a deletion unit, used to delete the source virtual character corresponding to the virtual character model in the virtual scene model template to obtain an intermediate virtual scene model template; an adding unit, used to play the intermediate virtual scene model template from a starting position, and add the virtual character model to the playback screen of the intermediate virtual scene model template; a control unit, used to control the movement of the virtual character model in the playback screen to form an action trajectory based on the character feature data, and record the action trajectory; a second generation unit, used to re-render the virtual character model in the intermediate virtual scene model template along the action trajectory to generate a second video containing the target virtual character.

[0090] Optionally, the control unit includes: a first control subunit, used to generate a control instruction for the virtual character model in the playback screen based on the character feature data, and add multiple virtual cameras corresponding to multiple perspectives of the playback screen, wherein the control instruction is used to instruct the virtual character model to interact with scene elements in the playback screen; a second control subunit, used to respond to the control instruction, control the virtual character model to move in the playback screen to form a first motion trajectory, and control the multiple virtual cameras to capture multiple groups of dynamic pictures of the virtual character model during the movement along the first motion trajectory.

[0091] Optionally, the second generating unit includes: an acquiring subunit, used to acquire multiple groups of dynamic images from the multiple virtual cameras; a generating subunit, used to select a group of video images from the multiple groups of dynamic images for each frame of the playback image of the intermediate virtual scene model template, and edit the playback images and the video images frame by frame to generate a second video containing the target virtual character.

[0092] Optionally, the control unit includes: an extraction subunit, used to extract facial features and body posture features in the character feature data in sequence according to a storage sequence, wherein the character feature data stores a plurality of groups of facial features and a plurality of groups of body posture features in a queue based on the storage sequence; a generation subunit, used to generate facial movement instructions and body movement instructions corresponding to the facial features and the body posture features, respectively, and use the facial movement instructions and the body movement instructions to synchronously control the movement of the target virtual image in the playback screen to form a second movement trajectory.

[0093] Optionally, the second acquisition module includes: a first processing unit, used to collect a first target audio, extract timbre features from the first target audio, and extract dialogue information of the virtual character model in the first video; and a generation unit, used to generate character dubbing data of the virtual character model based on the timbre features and the dialogue information.

[0094] Optionally, the second acquisition module includes: a second processing unit, used to collect second target audio and parse text information in the second target audio; a first configuration unit, used to configure the second target audio and the text information as character dubbing data and character dialogue data of the virtual character model, respectively; a second configuration unit, used to identify lip features and body movement features in the second target audio, and configure the lip features and the body movement features as lip movement data and body movement data of the virtual character model, respectively.

[0095] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0096] Example 3

[0097] The present application also provides an electronic device, Figure 6 is a structural diagram of an electronic device according to an embodiment of the present invention, such as Figure 6As shown, it includes a processor 61, a communication interface 62, a memory 63 and a communication bus 64, wherein the processor 61, the communication interface 62, and the memory 63 communicate with each other through the communication bus 64, and the memory 63 is used to store computer programs;

[0098] The processor 61 is used to implement the following steps when executing the program stored in the memory 63: loading a first video on a first client and detecting an editing request for the first video; in response to the editing request, obtaining source video template data of the first video, wherein the source video template data includes a virtual character model that allows editing; obtaining character feature data of the virtual character model; and based on the character feature data, rendering and generating a second video in a virtual scene model template corresponding to the source video template data.

[0099] Optionally, loading the first video on the first client includes: in a process of loading the virtual game on the first client, using screen frames of the virtual game to generate a game clip video; and loading the game clip video.

[0100] Optionally, obtaining character feature data of the virtual character model includes at least one of the following: obtaining first character feature data of a player-controlled character PCC model from the first client; obtaining second character feature data of a non-player character NPC model from the first client; obtaining third character feature data of a first virtual character model from the first client, and obtaining fourth character feature data of a second virtual character model from the second client.

[0101] Optionally, the character feature data includes third character feature data of the first virtual character model and fourth character feature data of the second virtual character model, and based on the character feature data, rendering and generating a second video in the virtual scene model template corresponding to the source video template data includes: calling the game scene sub-model, world motion sub-model, first virtual character sub-model, second virtual character sub-model, and game interaction sub-model of the virtual scene model template from the game rendering engine; using the third character feature data to edit the first virtual character sub-model in the game scene sub-model and the world motion sub-model, and using the fourth character feature data to edit the second virtual character sub-model in the game scene sub-model and the world motion sub-model to generate an intermediate video; using the game interaction sub-model to add interaction information between the first virtual character sub-model and the second virtual character sub-model in the intermediate video to generate the second video.

[0102] Optionally, the first client is a video social client, and the first video is loaded on the first client, and an editing request for the first video is detected, including: loading the first video on the video social client, and detecting the editing request for the first video in a playback interface of the video social client that plays the first video, wherein the editing request is carried in video frame data of the first video.

[0103] Optionally, after the first client loads the first video, the method further includes: extracting an editor tag from the first video, wherein the editor tag is used to identify an editing account associated with the first video; calculating traffic data generated by the first video on the first client; and saving the traffic data to a billing account of the editing account.

[0104] Optionally, saving the traffic data to the billing account of the editing account includes: extracting the parent account and child account of the first video from the hierarchical tree, wherein the parent account is the editing account of the source video of the first video, and the child account is the editing account of the first video; calculating the editing ratios of the parent account and the child account for the first video respectively; dividing the traffic data according to the editing ratios, and saving the divided traffic sub-data to the parent account and the child account respectively.

[0105] Optionally, obtaining the character feature data of the virtual character model includes: capturing at least one of the following of the virtual character model from the real scene data collected by the first client: three-dimensional facial feature data, action feature data, body feature data, and sound feature data; and / or requesting at least one of the following of the virtual character model from a feature material library: three-dimensional facial feature data, action feature data, body feature data, and sound feature data; selecting a game character in a virtual game, and reading the character feature data of the virtual character model from the character attributes or skill attributes of the game character.

[0106] Optionally, based on the character feature data, rendering and generating a second video in a virtual scene model template corresponding to the source video template data includes: deleting the source virtual character corresponding to the virtual character model in the virtual scene model template to obtain an intermediate virtual scene model template; playing the intermediate virtual scene model template from a starting position, and adding the virtual character model to the playback screen of the intermediate virtual scene model template; controlling the virtual character model to move in the playback screen to form an action trajectory based on the character feature data, and recording the action trajectory; and re-rendering the virtual character model in the intermediate virtual scene model template along the action trajectory to generate a second video containing the target virtual character.

[0107] Optionally, controlling the virtual character model to move in the playback screen to form an action trajectory based on the character feature data includes: generating a control instruction for the virtual character model in the playback screen based on the character feature data, and adding multiple virtual cameras corresponding to multiple perspectives of the playback screen, wherein the control instruction is used to instruct the virtual character model to interact with scene elements in the playback screen; in response to the control instruction, controlling the virtual character model to move in the playback screen to form a first motion trajectory, and controlling the multiple virtual cameras to capture multiple groups of dynamic pictures of the virtual character model during the movement along the first motion trajectory.

[0108] Optionally, re-rendering the virtual character model in the intermediate virtual scene model template along the action trajectory to generate a second video containing the target virtual character includes: obtaining multiple groups of dynamic images from the multiple virtual cameras; for each frame of the playback picture of the intermediate virtual scene model template, selecting a group of video pictures from the multiple groups of dynamic pictures, and editing the playback pictures and the video pictures frame by frame to generate a second video containing the target virtual character.

[0109] Optionally, controlling the virtual character model to move in the playback screen to form a motion trajectory based on the character feature data includes: extracting facial features and body posture features in the character feature data in sequence according to a storage sequence, wherein the character feature data stores a plurality of groups of facial features and a plurality of groups of body posture features in a queue based on the storage sequence; generating facial movement instructions and body movement instructions corresponding to the facial features and the body posture features, respectively, and using the facial movement instructions and the body movement instructions to synchronously control the target virtual image to move in the playback screen to form a second motion trajectory.

[0110] Optionally, obtaining the character feature data of the virtual character model includes: collecting a first target audio, extracting timbre features from the first target audio, and extracting dialogue information of the virtual character model in the first video; and generating character dubbing data of the virtual character model based on the timbre features and the dialogue information.

[0111] Optionally, obtaining the character feature data of the virtual character model includes: collecting a second target audio, parsing text information in the second target audio; configuring the second target audio and the text information as character dubbing data and character dialogue data of the virtual character model, respectively; identifying lip features and body movement features in the second target audio, and configuring the lip features and the body movement features as lip movement data and body movement data of the virtual character model, respectively.

[0112] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0113] The communication interface is used for communication between the above terminal and other devices.

[0114] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0115] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0116] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the method for generating video data described in any of the above embodiments.

[0117] In another embodiment provided by the present application, a computer program product including instructions is also provided, and when the computer is run on the computer, the computer executes the method for generating video data described in any one of the above embodiments.

[0118] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0119] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0120] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0122] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0124] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0125] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for generating video data, characterized in that: include: Loading a first video on a first client and detecting an editing request for the first video; In response to the editing request, acquiring source video template data of the first video, wherein the source video template data includes a virtual character model that allows editing; Acquiring character feature data of the virtual character model; According to the character feature data, a second video is rendered in a virtual scene model template corresponding to the source video template data, wherein the character feature data is used to configure or control the virtual character model in the virtual scene model template; Among them, after the first client loads the first video, it also includes: extracting an editor tag from the first video, wherein the editor tag is used to identify the editing account related to the first video; calculating the traffic data generated by the first video on the first client; saving the traffic data to the billing account of the editing account; wherein saving the traffic data to the billing account of the editing account includes: extracting the parent account and child account of the first video from the hierarchical tree, wherein the parent account is the editing account of the source video of the first video, and the child account is the editing account of the first video; calculating the editing ratios of the parent account and the child account to the first video respectively; dividing the traffic data according to the editing ratio, and saving the divided traffic sub-data to the parent account and the child account respectively.

2. The method according to claim 1, characterized in that Loading the first video on the first client includes: During the process of loading the virtual game on the first client, using the screen frames of the virtual game to generate a game clip video; Load the game clip video.

3. The method according to claim 1, characterized in that Acquiring the character feature data of the virtual character model includes at least one of the following: Acquire first character feature data of the player-controlled character PCC model from the first client; Acquire second character feature data of a non-player character NPC model from the first client; The third character feature data of the first virtual character model is obtained from the first client, and the fourth character feature data of the second virtual character model is obtained from the second client.

4. The method according to claim 1, characterized in that: The character feature data includes third character feature data of the first virtual character model and fourth character feature data of the second virtual character model. Based on the character feature data, rendering and generating a second video in a virtual scene model template corresponding to the source video template data includes: Calling the game scene sub-model, world motion sub-model, first virtual character sub-model, second virtual character sub-model, and game interaction sub-model of the virtual scene model template from the game rendering engine; Using the third character feature data to edit the first virtual character sub-model in the game scene sub-model and the world motion sub-model, and using the fourth character feature data to edit the second virtual character sub-model in the game scene sub-model and the world motion sub-model, to generate an intermediate video; The game interaction sub-model is used to add interaction information between the first virtual character sub-model and the second virtual character sub-model in the intermediate video to generate the second video.

5. The method according to claim 1, characterized in that The first client is a video social client, and loading a first video on the first client and detecting an editing request for the first video include: A first video is loaded in a video social client, and an editing request for the first video is detected in a playback interface of the video social client for playing the first video, wherein the editing request is carried in video frame data of the first video.

6. The method according to claim 1, characterized in that Acquiring the character feature data of the virtual character model includes: capturing at least one of the following of the virtual character model from the real scene data collected by the first client: three-dimensional facial feature data, action feature data, body feature data, and sound feature data; and / or, Requesting at least one of the following of the virtual character model from the feature material library: facial three-dimensional feature data, action feature data, body feature data, and voice feature data; A game character in a virtual game is selected, and character feature data of the virtual character model is read from character attributes or skill attributes of the game character.

7. The method according to claim 1, characterized in that Rendering and generating a second video in a virtual scene model template corresponding to the source video template data according to the character feature data includes: Deleting the source virtual character corresponding to the virtual character model from the virtual scene model template to obtain an intermediate virtual scene model template; Play the intermediate virtual scene model template from the starting position, and add the virtual character model to the playing screen of the intermediate virtual scene model template; Controlling the virtual character model to move in the playback screen to form a motion trajectory based on the character feature data, and recording the motion trajectory; The virtual character model is re-rendered in the intermediate virtual scene model template along the action trajectory to generate a second video containing the target virtual character.

8. The method according to claim 7, characterized in that Controlling the virtual character model to move in the playback screen to form an action trajectory based on the character feature data includes: Generating a control instruction for the virtual character model in the playback screen based on the character feature data, and adding a plurality of virtual cameras corresponding to a plurality of viewing angles of the playback screen, wherein the control instruction is used to instruct the virtual character model to interact with a scene element in the playback screen; In response to the control instruction, the virtual character model is controlled to move in the playback picture to form a first motion trajectory, and a plurality of virtual cameras are controlled to shoot a plurality of groups of dynamic pictures of the virtual character model during the movement along the first motion trajectory.

9. The method according to claim 8, characterized in that Re-rendering the virtual character model in the intermediate virtual scene model template along the action trajectory to generate a second video containing the target virtual character includes: Acquire multiple groups of dynamic images of the multiple virtual cameras; For each frame of the playback picture of the intermediate virtual scene model template, a group of video pictures is selected from the multiple groups of dynamic pictures, and the playback picture and the video picture are edited frame by frame to generate a second video containing the target virtual character.

10. The method according to claim 7, characterized in that Controlling the virtual character model to move in the playback screen to form an action trajectory based on the character feature data includes: Extracting facial features and body posture features in the character feature data in sequence according to a storage time sequence, wherein the character feature data stores a plurality of groups of facial features and a plurality of groups of body posture features in a queue based on the storage time sequence; Generate facial action instructions and body action instructions corresponding to the facial features and the body posture features respectively, and use the facial action instructions and the body action instructions to synchronously control the movement of the target virtual image in the playback screen to form a second action trajectory.

11. The method according to claim 1, characterized in that: Acquiring the character feature data of the virtual character model includes: Collecting a first target audio, extracting a timbre feature from the first target audio, and extracting the line information of the virtual character model in the first video; The character dubbing data of the virtual character model is generated based on the timbre characteristics and the dialogue information.

12. The method according to claim 1, characterized in that Acquiring the character feature data of the virtual character model includes: Collecting a second target audio, and parsing text information in the second target audio; configuring the second target audio and the text information as character dubbing data and character dialogue data of the virtual character model respectively; The lip shape features and body movement features in the second target audio are identified, and the lip shape features and the body movement features are configured as the lip shape movement data and the body movement data of the virtual character model, respectively.

13. A device for generating video data, characterized in that: include: A detection module, configured to load a first video on a first client and detect an editing request for the first video; A first acquisition module, configured to respond to the editing request and acquire source video template data of the first video, wherein the source video template data includes a virtual character model that allows editing; A second acquisition module, used to acquire character feature data of the virtual character model; A generating module, configured to render and generate a second video in a virtual scene model template corresponding to the source video template data according to the character feature data, wherein the character feature data is used to configure or control the virtual character model in the virtual scene model template; The device further includes: an extraction module, configured to extract an editor tag from the first video after the detection module loads the first video on the first client, wherein the editor tag is used to identify an editing account related to the first video; a calculation module, configured to calculate the traffic data generated by the first video on the first client; and a storage module, configured to save the traffic data to a billing account of the editing account; Among them, the saving module includes: an extraction unit, used to extract the parent account and child account of the first video from the hierarchical tree, wherein the parent account is the editing account of the source video of the first video, and the child account is the editing account of the first video; a calculation unit, used to calculate the editing ratios of the parent account and the child account to the first video respectively; a saving unit, used to divide the traffic data according to the editing ratio, and save the divided traffic sub-data to the parent account and the child account respectively.

14. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 12 when executed.

15. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 12.

Images