Video batch generation method, device, storage medium and program product

Abstract

Embodiments of the present application provide a video batch generation method, device, storage medium and program product. In the embodiments of the present application, an initial video template is subjected to variable processing to obtain materialized templates of multiple material types, the materialized templates are reorganized to obtain a video template required for generating a video, and then, according to a set of video materials corresponding to each video instance identifier, a target materialized template for reorganization is determined, and the target materialized template is reorganized in combination with a hierarchical relationship between the materialized templates to obtain a video template corresponding to each video instance identifier for respective video generation processing. Since the video template is obtained by personalized reorganization in combination with video materials for generating a video, the flexibility of video generation is improved. The reorganized video template has a corresponding relationship with the video instance identifier, the types of the video template are enriched, the style of each video generated in batch matches the corresponding video template, and the diversity of the video content is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, device, storage medium and program product for batch video generation. Background Technology

[0002] In the field of video generation, video editing software can be used to manually edit footage to obtain videos that meet individual needs. However, video editing software has a certain learning curve, and its use is labor-intensive, making it difficult to achieve mass video generation.

[0003] To lower the learning curve and improve video generation efficiency, some video editing software offers preset templates. The text or images in these templates can be replaced, allowing users to replace the text or images in the templates with their own footage to generate the desired video. This method of directly replacing text or images in preset templates simplifies the video generation process and is suitable for batch video production. However, videos generated in batches based on preset templates tend to have monotonous styles, suffer from severe homogenization, and lack flexibility in video generation. Summary of the Invention

[0004] This application provides a method, device, storage medium, and program product for batch video generation, which enriches the types of video templates so that each batch-generated video has its own video template, thereby improving the flexibility and diversity of batch-generated video content.

[0005] This application provides a method for batch video generation, comprising: responding to input operations on a video generation page regarding the number of videos and video categories, generating a batch video generation task, wherein the batch video generation task includes the number of videos N and video categories, where N is an integer ≥ 2; generating N video instance identifiers according to the batch video generation task, and generating a set of video materials related to the video category for each video instance identifier; obtaining multiple material templates obtained by variable processing of an initial video template, wherein the initial video template includes rendering rules for multiple video materials required to generate videos and the hierarchical relationship between the multiple video materials, and each material template is used to describe... A video material rendering rule; for each video instance identifier, based on the material type in a set of video materials corresponding to the video instance identifier, at least one target material template is determined from a plurality of material templates; based on the hierarchical relationship between the plurality of video materials included in the initial video template, the at least one target material template is combined to obtain a target video template corresponding to the video instance identifier, the target video template being used to describe the rendering rules and hierarchical relationship of the set of video materials; video generation processing is performed based on the target video templates corresponding to N video instance identifiers and a set of video materials to obtain N videos under the video category.

[0006] This application also provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor is coupled to the memory to execute the computer program to implement the steps in the various methods provided in this application.

[0007] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps in the methods described above.

[0008] This application also provides a computer program product, which includes a computer program / instructions that, when executed by a processor, enable the processor to implement the steps described in the above method embodiments.

[0009] In this embodiment, the initial video template is processed to obtain various material templates of different material types. These material templates are then recombined to obtain the video templates required for video generation. Furthermore, each video instance identifier corresponds to a video generation and is bound to its own set of video materials. Based on the set of video materials corresponding to the video instance identifier, a target material template for recombination is determined. Combining the hierarchical relationship between the material templates, the target material template is recombined to obtain the video templates corresponding to each video instance identifier for their respective video generation processing. Since the video templates are obtained by personalized recombination of the video materials of the generated video, the flexibility of video generation is improved. Moreover, the recombined video templates have a corresponding relationship with the video instance identifiers, which enriches the types of video templates to a certain extent, making the style of each batch-generated video match the corresponding video template, thereby improving the diversity of batch-generated video content. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0011] Figure 1 A flowchart illustrating a method for batch video generation provided as an exemplary embodiment of this application;

[0012] Figure 2 This is an interactive schematic diagram of a video batch generation method in another exemplary embodiment of this application;

[0013] Figure 3 This is a schematic diagram of the structure of an electronic device provided as another exemplary embodiment of this application. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] It should be noted that, in the cases involving user information in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards.

[0016] Additionally, it should be noted that when user interaction operations or triggering operations are involved in the embodiments of this application, these operations include, but are not limited to, various interaction methods such as touch operations, gesture operations, voice operations, head movement operations, and eye movement operations. Touch operations include, but are not limited to, click operations, double-click operations, long-press operations, swipe operations, pinch operations, or mouse hover operations. Swipe operations include, but are not limited to, straight-line swipes and curved-line swipes.

[0017] Furthermore, it should be noted that, in the cases where the embodiments of this application involve jumping between the first interface and the second interface, the jumping methods involved in the embodiments of this application include, but are not limited to: jumping directly from the first interface to the second interface, or jumping from the first interface to the task interface and completing the corresponding task operation on the task interface before jumping to the second interface; completing the corresponding task operation on the task interface includes, but is not limited to: completing the game operation on the game interface when the task interface is implemented as a game interface; completing identity authentication on the identity authentication interface when the task interface is implemented as an identity authentication interface; completing the recharge operation on the recharge interface when the task interface is implemented as a recharge interface; and so on.

[0018] To address the technical problems of monotonous and homogeneous video styles and poor flexibility in video generation caused by batch generation from preset templates, this application embodiment obtains various material-type templates through variable processing of the initial video template. These material-type templates are then recombined to obtain the video templates required for video generation. Furthermore, each video instance identifier corresponds to a video generation and is bound to its own set of video materials. Based on the set of video materials corresponding to the video instance identifier, a target material-type template for recombination is determined. Combining the hierarchical relationship between the material-type templates, the target material-type template is recombined to obtain the video templates corresponding to each video instance identifier for their respective video generation processing. Since the video templates are obtained by personalized recombination of the video materials of the generated video, the flexibility of video generation is improved. Moreover, the recombined video templates have a corresponding relationship with the video instance identifiers, which enriches the types of video templates to a certain extent, making the style of each batch-generated video match the corresponding video template, thereby improving the diversity of batch-generated video content.

[0019] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0020] Figure 1 This is a flowchart illustrating a method for batch video generation, provided as an exemplary embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0021] S101: Respond to the input operations on the video generation page regarding the number of videos and video categories, and generate a batch video generation task. The batch video generation task includes the number of videos N and the video categories, where N is an integer ≥ 2.

[0022] S102: Based on the batch video generation task, generate N video instance identifiers, and generate a set of video materials related to the video category for each video instance identifier;

[0023] S103: Obtain multiple material templates obtained by variable processing of the initial video template. The initial video template includes the rendering rules of multiple video materials required to generate the video and the hierarchical relationship between the multiple video materials. Each material template is used to describe the rendering rules of a video material.

[0024] S104: For each video instance identifier, based on the material type in a set of video materials corresponding to the video instance identifier, determine at least one target material template from multiple material templates; based on the hierarchical relationship between the multiple video materials included in the initial video template, combine at least one target material template to obtain the target video template corresponding to the video instance identifier. The target video template is used to describe the rendering rules and hierarchical relationship of a set of video materials.

[0025] S105: Based on the target video templates corresponding to each of the N video instance identifiers and a set of video materials, perform video generation processing to obtain N videos under the video category.

[0026] In this embodiment, the executing entity of the above-described video batch generation method is not limited. For example, the method can be implemented as a service product, which can adopt a client-server architecture. In the case of a client-server architecture, on the one hand, the client provides a video generation page for batch video generation to the user, receiving the user's input on the number and category of videos, and then initiating a batch video generation task to the server. On the other hand, the server responds to the client's initiation of a batch video generation task through the video generation service page, and uses the server's computing resources, network bandwidth, storage resources, and other resources to generate videos in batches, which helps to improve the speed of video generation.

[0027] For example, as the processing power of client-side hardware increases, the above method can also be executed by the client. The client can provide a video generation page to the user and respond to the user's input on the number and category of videos on the video generation page, generating batch video generation tasks. Then, the client's own computing and storage resources are used to generate videos in batches. In this case, when the batch generation tasks are deployed and executed on the client side, there is no need to transmit data to the server, which can save network latency.

[0028] In this embodiment, the batch video generation task includes the number of videos N and video categories, where N is an integer ≥ 2. The video category describes the theme of the generated video content; the specific implementation of the video category is not limited. For example, it includes, but is not limited to: product introductions, tutorials, science popularization, and beauty and skincare, etc. Optionally, the video category can be implemented as a single-level video category, such as business registration or legal consultation. Optionally, the video category can also be implemented as a multi-level video category. For example, the first-level video category could be business registration; the second-level video categories of this first-level category could be cleaning or food business, etc.

[0029] In this embodiment, N video instance identifiers are generated according to the batch generation task. Each of the N video instance identifiers is different. Each video instance identifier can be used to uniquely represent a video to be generated, so as to facilitate the tracking of the video materials and video templates required for the video to be generated. That is to say, the video instance identifier can also serve as a unique identifier for the relevant content (such as video materials) of the video to be generated.

[0030] The method for generating the N video instance identifiers is not limited. For example, it can include, but is not limited to, numbers and strings. For instance, it can start from any integer and form an increasing sequence with a fixed step size to obtain N integers as the N video instance identifiers; or it can be a preset set of N strings, etc.

[0031] In this embodiment, a set of video materials related to a video category is generated for each video instance identifier. This set of video materials is used to generate the video corresponding to that video instance identifier. Each set of video materials includes video materials of at least one material type. In some embodiments of this application, video materials of one material type are simply referred to as one type of video material.

[0032] The material type refers to the type of media resources that constitute the video, including but not limited to: audio, video, background images, subtitles, digital humans, and other material types.

[0033] In this embodiment, the implementation method of generating a set of video materials related to the video category for each video instance is not limited.

[0034] In one optional implementation, for any video instance identifier, video materials can be randomly extracted from multiple material types stored in the basic material library. At least one extracted video material is then used as a set of video materials for that video instance identifier. The basic material library stores multiple video materials under various video categories.

[0035] In another optional implementation, for any video instance identifier, the semantic similarity between at least one video material in the basic material library and the video category is calculated to obtain multiple similarity information; the video materials that meet the similarity conditions among the multiple similarity information are taken as a group of video materials for that video instance identifier.

[0036] Furthermore, in this embodiment, multiple material templates are obtained by performing variable processing on the initial video template. Optionally, the initial video template can be implemented as an MLT (Media Lovin' Toolkit) template. An MLT template is an open-source framework for multimedia processing. It is an XML-based file that records all parameters for video editing, such as video clips on the timeline, audio tracks, filter effects, transitions, etc., and can be used for video editing.

[0037] In this embodiment, the initial video template can be an existing video production framework. This framework can be pre-defined or exported from video editing software. This embodiment does not limit the specific method of obtaining the initial video template. For example, the initial video template can be a pre-defined general video template provided by the system, a custom video template created by the user according to their needs, or a video template imported from other external sources.

[0038] The initial video template includes rendering rules for various video materials required to generate the video, as well as the hierarchical relationships between these video materials. In this embodiment, the rendering rules for each type of video material describe the rendering logic followed by that type of video material during the video rendering process, in order to achieve the desired rendering effect in the generated video. The hierarchical relationship between the various video materials within the initial video template refers to the layer stacking order of the various video materials in the initial video template during the video generation process, used to determine the overlapping and occlusion logic of the various video materials in video generation. For example, in generating a video, subtitles can be located on a relatively upper layer, the background image can be located on a relatively lower layer, and the digital human can be superimposed on the background image but below the subtitle layer.

[0039] In this embodiment, the variable processing of the initial video template is partly reflected in the structured decomposition of the initial video template and the templated rendering parameter information based on the material type as the splitting variable, to obtain multiple material-based templates. There is a correspondence between the material-based templates and the material types. For example, material types include audio, video, background images, subtitles, digital humans, etc. The corresponding material-based template types include, but are not limited to: audio material-based templates, video material-based templates, background image material-based templates, subtitle material-based templates, digital human material-based templates, etc. In other words, each material type corresponds to one material-based template, and each material-based template describes the rendering rules for that type of video material.

[0040] In this embodiment, the timing of the variable processing is not limited. For example, the initial video template can be variableized in advance. Alternatively, the initial video template can be variableized dynamically. For details on how to perform variable processing, please refer to subsequent embodiments.

[0041] In this embodiment, the variable-processed material templates can be modularly recombined. For each video instance identifier, based on the material types in a set of video materials corresponding to that video instance identifier, at least one target material template is determined from multiple material templates. There is a correspondence between the material types in this set of video materials and the material templates.

[0042] For example, if the set of video materials includes audio, video, background images, and subtitles, then the target material template can include the target material templates corresponding to each of the audio, video, background images, and subtitles. Similarly, if the set of video materials includes audio, video, background images, subtitles, and digital human materials, then the target material template can include the target material templates corresponding to each of the audio, video, background images, subtitles, and digital human materials.

[0043] Furthermore, given at least one target material template corresponding to a set of video materials, based on the hierarchical relationship between the various video materials included in the initial video template, the at least one target material template is combined to obtain the target video template corresponding to the video instance identifier. The target video template describes the rendering rules and hierarchical relationship of a set of video materials.

[0044] The initial video template includes a hierarchical relationship among various video materials. This hierarchy represents the layer stacking order of multiple material templates and can be used to organize and integrate target material templates, thereby forming a target video template with a clear hierarchy and corresponding rendering rules for the video materials. The target video template describes the hierarchical relationship of a group of video materials, which is consistent with the hierarchical relationship of that group of video materials in the initial video template.

[0045] In this embodiment, each of the N video instance identifiers corresponds to its own target video template. That is, each group of video materials has its own corresponding target video template, enriching the variety of target video templates used in batch video generation. Based on the rendering rules described by the N target video templates, video generation processing is performed on the grouped video materials corresponding to the N video instance identifiers, ensuring that the style of each batch-generated video matches the adopted video template, thereby increasing the diversity of the batch-generated video content.

[0046] Given the target video templates corresponding to each video instance identifier, video generation processing is performed based on the target video templates corresponding to each of the N video instance identifiers and a set of video materials to obtain N videos under the video category. Video generation processing refers to the process of filling and rendering the target video templates with video materials based on the target video templates corresponding to each video instance identifier and a set of video materials to obtain the video corresponding to that video instance identifier. For example, for N video instance identifiers, the set of video materials corresponding to each of the N video instance identifiers can be filled into N target video templates to obtain filled target video templates. Then, the filled target video templates can be rendered to obtain the videos under the video category.

[0047] In one optional embodiment, the video generation corresponding to each of the N video instance identifiers can be performed in batches. Each batch of video generation processes M videos in parallel, where M is less than N and M is an integer. After the M videos are generated, the process continues to generate the next M videos until all the videos corresponding to the N video instance identifiers have been processed. By performing batch video generation, the resource utilization of the server is improved, while avoiding server overload caused by high concurrency.

[0048] In this embodiment, during batch video generation, various material templates of different types are obtained by variable processing of the initial video template. These material templates are then recombined to obtain the video templates required for video generation. Furthermore, for each video generation, a corresponding video instance identifier is generated. Each video instance identifier is bound to a corresponding set of video materials. Based on the set of video materials corresponding to the video instance identifier, a target material template for recombination is determined from various material templates. Combining the hierarchical relationship between the material templates provided by the initial template, the target material template is organized and integrated to obtain the video template corresponding to each video instance identifier. This template is then used for video generation processing of the set of video materials corresponding to each video instance identifier. Since the video templates can be personalized by combining the video materials of the generated video, the flexibility of video generation is improved. Additionally, the recombined video templates have a corresponding relationship with the video instance identifiers, enriching the variety of video templates to a certain extent, so that the style of each batch-generated video matches the adopted video template, thus increasing the diversity of batch-generated video content.

[0049] The following section introduces the methods for handling variables.

[0050] In one optional embodiment, the step of variableizing the initial video template includes: obtaining the initial video template, which includes rendering rules for various video materials required to generate the video and the hierarchical relationship between the various video materials; parsing the initial video template using material type as a splitting variable to obtain information fragments corresponding to various material types; extracting rendering parameter information corresponding to each of the various material types from the information fragments corresponding to each of the various material types; and templating the rendering parameter information corresponding to each of the various material types to obtain various materialized templates, each materialized template being used to describe the rendering rules of a video material.

[0051] Furthermore, after obtaining the initial video template, the template is parsed using the material type as a splitting variable. Here, material type refers to the different categories of video materials, used as a classification standard to distinguish different video materials, while the splitting variable refers to the independent information segments obtained by dividing the initial video template according to the material type during parsing. For example, if the initial video template contains both image and audio video materials, it will be split into independent image and audio information segments based on the material type, and then processed separately.

[0052] Here, an information fragment refers to an information segment extracted from the initial video template that corresponds to the material type. An information fragment can contain rendering parameter information for the corresponding material type. For example, an information fragment can contain rendering parameter information such as path information, playback time, and effects.

[0053] In this embodiment, rendering parameter information describes at least one attribute of a material file of a certain material type. Each attribute can be used as a rendering parameter, and the attribute value of each attribute in the initial video template can be used as the default parameter value of the corresponding rendering parameter. Rendering parameters with default parameter values ​​form a rendering parameter information corresponding to the material file of that material type. Here, "material file" refers to various resource files used in the materialization template, mainly including various video materials, such as images, videos, and audio. Rendering parameter information can be used to control the rendering logic of the material file so that the material file can achieve the rendering effect presented in the final generated video. Rendering parameter information quantifies and describes the attributes of the material file, enabling the conversion of video material attributes into rendering parameter information attributes, and controlling the rendering effect of the video material in the final generated video through the rendering parameter information. In other words, the attributes of the video material are described by the rendering parameter information as corresponding rendering parameters, and rendering parameters with default parameter values ​​define the attribute value of a certain attribute of the material file.

[0054] The rendering parameter information can be extracted from the information fragments corresponding to each of the various material types. The rendering parameter information for each material type includes at least one rendering parameter and its corresponding default parameter value. Each rendering parameter describes the attributes of a material file of a certain material type. In this embodiment, the specific attributes corresponding to the rendering parameters for each of the various material types are not limited.

[0055] In this embodiment, templating can transform the rendering parameter information corresponding to various material types from fixed-configuration default parameter values ​​into a set of variable parameters with dynamically replaceable parameter values. Specifically, templating the rendering parameter information corresponding to various material types yields multiple material templates for each material type. Each material template describes the rendering rules for a particular material type, and these rules include at least one variable parameter. Each variable parameter is associated with multiple candidate parameter values ​​to control the rendering effect of the generated video.

[0056] In this embodiment, the material template is the result of templated processing of rendering parameter information corresponding to various material types. It includes path information placeholders for the material files, as well as variable parameters and their associated candidate parameter values. The material template describes the rendering rules of the corresponding material type in the generated video. The rendering rules describe the rendering logic followed by the video material during the video rendering process, thereby controlling the expected rendering effect of the material file in the generated video. Specifically, by transforming the rendering parameter information from fixed-configuration rendering parameters with default values ​​into a material template formed by a dynamically replaceable set of variable parameters, different candidate parameter values ​​can be assigned to the variable parameters to flexibly adjust the rendering effect of the material files in the generated video in different scenarios. The templated rendering parameter information forms multiple modularly recombinable material templates. Different target video templates can be recombined for different material types, enabling the batch generation of diverse videos without the need for separate video template design and adjustment for each video. This not only saves time and manpower but also improves the flexibility and content diversity of video template generation, thereby achieving efficient batch generation of diverse style videos.

[0057] For example, for the same material template, by assigning different candidate parameter values ​​to the variable parameters, the attribute values ​​of various properties corresponding to the material type of the material template can be flexibly set, thereby achieving fine-grained control over the rendering rules of the material files. Material templates improve the flexibility of video generation, significantly improve the efficiency and quality of video generation, and meet diverse creative needs.

[0058] In this embodiment, it is not limited to the specific content of the candidate parameter values ​​associated with the variable parameters obtained after the rendering parameter information corresponding to various material types is templated.

[0059] By associating multiple candidate parameter values ​​with variable parameters, the variable parameters can be flexibly adjusted according to needs, improving the flexibility and diversity of video template generation, and thus achieving efficient batch generation of videos with diverse styles.

[0060] In one optional embodiment, the initial video template is parsed using the material type as a splitting variable to obtain information fragments corresponding to various material types. This includes: loading the XML document corresponding to the initial video template, where the XML document includes a root element and multiple non-root elements connected to the root element. Each non-root element includes multiple specific elements, each describing the rendering rules for a particular material type; traversing the non-root elements in the XML document starting from the root element to identify the multiple specific elements; and extracting multiple information fragments containing the multiple specific elements as information fragments corresponding to the various material types. The initial video template's XML document can be loaded into memory to form a parsable tree structure. In this embodiment, a DOM (Document Object Model) parser can be used to load the XML document into memory, forming a tree-structured parsing method.

[0061] The tree structure includes a root element and non-root elements. The root element is the top-level node of the XML document, and non-root elements are directly or indirectly nested under the root element. An XML document has one and only one root element, which is the first element of the XML document and can be used as the starting point of the XML document.

[0062] Non-root elements are child elements directly or indirectly nested under the root element, used to divide the initial video template into different information segments according to the material type. Non-root elements include multiple specific elements. By traversing the non-root elements in the XML document starting from the root element, multiple specific elements can be identified. Specific elements are elements that directly describe the rendering rules for a certain type of material, and each specific element corresponds to a specific material type.

[0063] By extracting these information fragments, the system can quickly identify the attributes of the source files and render them based on their attribute values. This design for extracting information fragments significantly improves the flexibility and diversity of video generation, meeting diverse creative needs, and providing a technological foundation for efficient batch video generation.

[0064] In an optional embodiment, starting from the root element, the non-root elements in the XML document are traversed to identify multiple specific elements, including: S1, starting from the root element, traversing the non-root elements in the XML document; S2, for the currently traversed non-root element, obtaining the element tag contained in the currently traversed non-root element; S3, if the element tag is a specific tag, determining whether the currently traversed non-root element contains child elements; S4, if the currently traversed non-root element contains child elements, then the child element is taken as the currently traversed non-root element, and the execution step S2 is returned; S5, if the element tag is a non-specific tag, then the next non-root element is encountered, and the execution step S2 is returned; S6, if the currently traversed non-root element does not contain child elements, then the currently traversed non-root element is taken as a specific element.

[0065] In step S1, the non-root elements of the XML document are traversed one by one, starting from the root element. In this embodiment, the specific implementation strategy of the traversal is not limited. For example, the traversal can be implemented using a depth-first search algorithm or a breadth-first search algorithm. In this embodiment, the process of traversing the non-root elements in the XML document to identify multiple specific elements is described in detail using a depth-first search algorithm as an example.

[0066] Next, step S2 is executed. For the currently traversed non-root element, the element tags contained in the currently traversed non-root element are obtained. In an XML document, element tags are identifiers enclosed in angle brackets (<>), used to mark the type and semantic meaning of an element. Different material types are distinguished by the name of the element tags; the specific location of the material is located by the hierarchical relationship of the element tags.

[0067] After retrieving the element tags contained in the currently traversed non-root element, determine whether the element tag is a specific tag. The specific tag can be a predefined set of key tags representing element tags in the XML document that require special handling, used to identify the type of material to be extracted.

[0068] Next, proceed to either step S3 or S5. If step S5 is executed, i.e., the element tag is a non-specific tag, then proceed to the next non-root element and return to step S2.

[0069] If step S3 is executed, i.e., the element tag is a specific tag, then it continues to determine whether the currently traversed non-root element contains child elements. Here, child elements can be other elements nested within the currently traversed non-root element.

[0070] Next, execute either step S4 or S6. If step S4 is executed, meaning the currently traversed non-root element contains child elements, then the child element is treated as the currently traversed non-root element, and the process returns to step S2. If step S6 is executed, meaning the currently traversed non-root element does not contain child elements, then the currently traversed non-root element is treated as a specific element. Step S4 is the recursive processing when child elements are present; the child element of the currently traversed non-root element is set as the new traversal starting point, and steps S2-S6 are re-executed for each child element. After the recursion ends, the traversal of other non-root elements continues. Step S6 is the end-collection when there are no child elements; the currently traversed non-root element is treated as a specific element.

[0071] In one optional embodiment, rendering parameter information corresponding to each of the various material types is extracted from information fragments corresponding to each material type, including: for each information fragment, extracting the path information of the material file and at least one attribute value of the material file from the information fragment, the attribute value being used to render the material file; reading the material file according to the path information, determining the material type described by the information fragment according to the extension of the material file; using at least one attribute to which at least one attribute value belongs as at least one rendering parameter, and using at least one attribute value as the default parameter value of at least one rendering parameter, so as to obtain the rendering parameter information corresponding to the material type described by the information fragment.

[0072] The information fragment is a portion extracted from the XML document that is related to the material type, containing the path information of the material file and at least one attribute value of the material file. The material file can be used to construct various media files for the final generated video. In this embodiment, the material file may include, but is not limited to: files in formats such as MP4 and AVI, which contain moving images and audio, and are video files used to display a series of continuous images; and files in formats such as WAV and MP3, which provide audio files containing background music, narration, or special effects sounds.

[0073] In this context, at least one attribute value extracted from the source file is a specific value corresponding to the attribute, describing the specific state of the attribute. Attributes can describe the rendering effect of the source file. For example, for a video file, attributes can be video duration, video speed, filter effects, etc. If the attribute is video duration, the attribute value can be 2 minutes, 1 hour, 1 day, etc.

[0074] In one optional embodiment, the rendering parameter information corresponding to the material type described by the information fragment includes at least one rendering parameter and a default parameter value for at least one rendering parameter. The default parameter value can be the attribute value of the attribute corresponding to the rendering parameter before quantization. Specifically, at least one attribute extracted from the material file in the information fragment can be used as at least one rendering parameter, and the at least one attribute value can be used as the default parameter value for at least one rendering parameter to obtain the rendering parameter information. That is, the rendering parameter and the default parameter value are combined into a key-value pair set to control the rendering logic of the material file in the video to achieve the desired rendering effect.

[0075] In one optional embodiment, the rendering parameter information corresponding to various material types is templated to obtain multiple material templates. This includes: for each material type, selecting at least one variable parameter from the rendering parameter information corresponding to that material type; associating multiple candidate parameter values ​​with the at least one variable parameter; and generating a material template corresponding to the material type based on the multiple candidate parameter values ​​associated with the at least one variable parameter. The variable parameter is obtained by variability processing of rendering parameters with default values. Each variable parameter is associated with multiple candidate parameter values, and different candidate parameter values ​​correspond to different rendering logics to produce different rendering effects in the generated video. This can be used to generate diverse material templates. For each material type, at least one variable parameter is selected from its corresponding rendering parameter information. These variable parameters can vary within an adjustment range to obtain different material templates. For example, when the material file is video material, playback speed and filter effects can be selected as variable parameters. Associating the variable parameter with multiple candidate values ​​limits the adjustment range of the variable parameter, ensuring that the material file of the generated video meets the design requirements and avoiding invalid settings.

[0076] In one optional embodiment, when selecting at least one variable parameter from the rendering parameter information corresponding to the material type, two selection methods are provided: one is to treat all rendering parameters as variable parameters, and the other is to select some rendering parameters as variable parameters based on weight values. When selecting to treat all rendering parameters as variable parameters, there is no need to compare weight values, and all rendering parameters can be adjusted as variable parameters.

[0077] In one optional embodiment, selecting some rendering parameters as variable parameters based on weight values ​​can be achieved by selecting at least one variable parameter from the rendering parameter information corresponding to the material type. This includes: pre-configuring weight values ​​for each rendering parameter for the material type; parsing each rendering parameter from the rendering parameter information corresponding to the material type; and selecting at least one rendering parameter whose weight value is greater than a set weight threshold as at least one variable parameter. The weight value can serve as an importance score assigned to each rendering parameter, quantifying the influence of the rendering parameter on the rendering effect of the material file in video generation. This allows for the selection of at least one rendering parameter that has a significant impact on user perception or application goals as at least one variable parameter. The weight threshold is a predefined critical value. Users can filter rendering parameters with weight values ​​higher than the weight threshold, selecting parameters with weight values ​​higher than the weight threshold as variable parameters. This avoids using too many irrelevant rendering parameters as variable parameters, preventing configuration conflicts or rendering logic confusion.

[0078] In this embodiment, the method of pre-configuring the weight values ​​of each rendering parameter is not limited. For example, the weight values ​​of each rendering parameter can be pre-configured based on empirical annotation or automatically generated through user behavior data analysis.

[0079] Optionally, selecting at least one variable parameter from the rendering parameter information corresponding to the material type can also be done randomly based on the set number of variable parameters. In an optional embodiment, generating a materialized template corresponding to the material type based on multiple candidate parameter values ​​associated with at least one variable parameter includes: adding each rendering parameter and its default value to a preset template file corresponding to the material type; adding multiple candidate parameter values ​​associated with at least one variable parameter to the preset template file; and adding a placeholder to the preset template file to carry the material file corresponding to the material type, thereby obtaining the materialized template corresponding to the material type. The preset template file is a basic video template containing basic configurations, including the basic structure of the video template and some preset rendering parameters. Adding each rendering parameter and its default value to the preset template file corresponding to the material type ensures that the final materialized template has complete rendering parameter information.

[0080] In this embodiment, in addition to adding the rendering parameters corresponding to each material type and their default values ​​to the preset template file, at least one variable parameter associated with multiple candidate parameter values ​​is added to the preset template file. These candidate parameter values ​​offer a variety of choices, allowing the user or system to select different candidate parameter values ​​according to their needs.

[0081] In one optional embodiment, based on adding at least one variable parameter associated with multiple candidate parameter values ​​to a preset template file, placeholders can be added to the preset template file to carry the material files corresponding to the material type, thereby obtaining a materialized template corresponding to the material type. Here, the placeholder is a reserved position used to fill in the material file, supporting dynamic replacement; for example, the path information of the actual material file corresponding to the material type can be filled in. By using placeholders, different material files can be flexibly replaced without modifying the template structure.

[0082] In the above embodiments, the material template integrates the rendering parameters of the material type and their default parameter values ​​into a preset base template, ensuring the integrity of the rendering parameter information of the generated material template. At the same time, by associating multiple candidate parameter values ​​with variable parameters, the parameter values ​​of variable parameters are dynamically selected to flexibly adjust to correspond to the video style. The embedding of placeholders further decouples the parameter configuration of the initial video template from the material files, supporting the dynamic replacement of material files without modifying the template structure. Thus, it allows the assignment of corresponding candidate parameter values ​​to variable parameters according to application requirements to obtain freely combinable material templates. Ultimately, this significantly improves the flexibility and diversity of video template generation, thereby efficiently mass-producing video content with different styles and solving the problems of cumbersome parameter configuration, complex adaptation process, and serious homogenization of generated video content in traditional video production.

[0083] Based on the aforementioned multiple source templates, batch video generation or single video generation can be performed using these templates. For batch video generation scenarios, the multiple source templates provided in this application embodiment can generate multiple videos with diverse styles and content. The following is a detailed description of one implementation method for batch video generation based on the multiple source templates provided in this application embodiment.

[0084] In this embodiment, the initial video template is variableized to obtain multiple material templates. During batch video generation, for each video instance identifier, at least one target material template is determined from the multiple material templates based on the material type in a set of video materials corresponding to the video instance identifier.

[0085] In one optional embodiment, when determining at least one target material template from multiple material templates based on the material type in a set of video materials corresponding to a video instance identifier, the method includes: identifying at least one material type contained in a set of video materials corresponding to a video instance identifier; selecting at least one initial material template from multiple material templates based on the at least one material type contained in the set of video materials, wherein each material type corresponds to one initial material template; and adjusting the parameters of at least some of the selected initial material templates to obtain at least one target material template.

[0086] In this embodiment, the material templates included in the multiple material templates obtained by variable processing of the initial video template are referred to as initial material templates. When at least one initial material template corresponding to a certain group is determined from the multiple initial material templates, the rendering rules described by the initial material templates can be referred to as initial rendering rules. Furthermore, parameters are adjusted for at least some of the initial material templates to obtain at least one target material template. Each initial material template, after parameter adjustment, yields a corresponding target material template. The target material template after parameter adjustment contains target rendering rules, which are different from the initial rendering rules described by the initial material templates.

[0087] In this embodiment, since the initial material template is obtained through variableization, the variableization process can transform the fixed rendering parameters in the initial video template into variable parameters that can be dynamically assigned values, thereby achieving flexible configuration of the template content. Different values ​​of the optional parameters will result in different rendering rules. Optionally, each initial material template includes at least one variable parameter, and each variable parameter is associated with multiple candidate parameter values ​​and default parameter values.

[0088] In this embodiment, parameter adjustment of the initial material template is performed to assign different candidate parameter values ​​to the variable parameters of the initial material template, thereby obtaining multiple target material templates with different candidate parameter values. A target material template for a specific material type can be used to render video materials grouped from different video instance identifiers. Different variable parameter values ​​indicate different rendering rules described by the target material templates for the same material type, resulting in different rendering results for batch-generated videos, creating differentiation between different videos, and enhancing the richness of video content. The following will describe how to adjust the parameters of the initial material template.

[0089] In one optional embodiment, adjusting the parameters of at least a portion of the initial material templates in at least one initial material template to obtain at least one target material template includes: determining the number of templates whose parameters need to be adjusted, wherein the number of templates is less than or equal to the number of at least one initial material template; selecting initial material templates to be adjusted from the at least one initial material template according to the number of templates to be adjusted; determining variable parameters to be adjusted from the initial material templates to be adjusted; randomly determining a target parameter value from a plurality of candidate parameter values ​​associated with the variable parameter to be adjusted; and assigning the target parameter value to the variable parameter to be adjusted to obtain the target material template.

[0090] In this embodiment, the method for determining the number of templates whose parameters need to be adjusted is not limited. For example, parameters can be adjusted for each initial video template, in which case the number of templates to be adjusted is the number of at least one initial material template. Alternatively, a random integer within a preset range can be generated using a random number generation algorithm as the number of templates to be adjusted. The preset range refers to the number of templates being less than or equal to the number of at least one initial material template. The random number generation algorithm is not limited and includes, but is not limited to, methods such as the Linear Congruential Generator (LCG) and the Mersenne Twister algorithm.

[0091] Further, based on the number of templates to be adjusted, initial material templates to be adjusted are selected from at least one initial material template. In an optional embodiment, the initial material templates to be adjusted are selected based on the priority of the material type and the number of templates to be adjusted. The priority of the material type refers to the importance of the material type to the presentation effect of the generated video. For example, subtitle material types generally have a lower importance to the presentation effect, so the priority of subtitle material types can be set to a lower priority; relatively speaking, the priority of background images can be higher than that of subtitles, so they can be set to a medium priority; and digital humans have a higher importance to the presentation effect, so they can be set to a higher priority. Furthermore, based on the priority, templates for adjusting parameters can be selected first from initial material templates with higher material type priority. If the number of these initial material templates is less than the previously determined number of templates to be adjusted, then initial material templates with lower material type priority can be selected, and the final number of selected templates to be adjusted is less than or equal to the number of at least one initial material template.

[0092] Furthermore, from the initial material template to be adjusted, the variable parameters to be adjusted are determined. In one optional embodiment, all variable parameters in the initial material template to be adjusted can be used as the variable parameters to be adjusted. In another optional embodiment, a target variable parameter in the initial material template to be adjusted is used as the optional parameter to be adjusted, and the target variable parameter is a pre-selected optional parameter.

[0093] Then, the target parameter value is randomly determined from multiple candidate parameter values ​​associated with the variable parameter to be adjusted. In some embodiments, the N video instance identifiers each have at least one initial material template with the same variable parameter to be adjusted. The multiple candidate parameter values ​​associated with the variable parameter to be adjusted can be randomly selected as the target parameter values ​​for the N video instance identifiers, so that the optional parameter values ​​of the N video instance identifiers are as different as possible.

[0094] Furthermore, the target parameter value is assigned to the variable parameter to be adjusted to obtain the target material template.

[0095] Once the target material template is obtained, it is combined to obtain the target video template. This embodiment does not limit the combination method. Two combination methods are provided below, but are not limited to these.

[0096] In one optional embodiment, a basic video template is generated based on the hierarchical relationship between various video materials included in the initial video template. This basic video template serves as the framework for the target video template and includes multiple blank structural positions corresponding to various video materials. The blank structural positions are placeholders for target material templates preset in the basic video template, used to identify the positions where target material templates can be inserted. Each blank structural position corresponds to the filling of a target material template of a certain material type. The positional relationship between the multiple blank structural positions reflects the hierarchical relationship between various video materials. As described in the above embodiment, the hierarchical relationship represents the order in which various materials are superimposed in the generated video. In some embodiments, this hierarchical relationship is extracted from the initial video template; alternatively, it can be preset based on at least one target material template, meaning the hierarchical relationship can be set as needed. For example, the structural position corresponding to subtitles is located at the top layer, the structural position corresponding to digital humans is located at the middle layer, and the structural position corresponding to background images is located at the bottom layer. Further, at least one target material template is inserted into the corresponding blank structural positions in the basic video template to obtain the target video template corresponding to the video instance identifier.

[0097] In another optional embodiment, based on at least one target material template, the structural bits containing the rendering rules of video materials of the same material type in the initial video template are overwritten to obtain the target video template corresponding to the video instance identifier. The difference between the structural bits and the aforementioned blank structural bits is that each type of video material's rendering rule occupies one structural bit in the initial video template, while blank structural bits are empty. The positional relationship between the structural bits reflects the hierarchical relationship between multiple video materials.

[0098] Given a target video template, video generation processing is performed based on the target video template corresponding to each video instance identifier and a set of video materials. In an optional embodiment, when performing video generation processing based on the target video templates corresponding to N video instance identifiers and a set of video materials to obtain N videos under the video category, the process includes: for each video instance identifier, filling the set of video materials corresponding to the video instance identifier into the target material template of the target video template corresponding to the video instance identifier; for the filled target video template, rendering the set of video materials according to the rendering rules and hierarchical relationships of the set of video materials described in the filled target video template to obtain a video under the video category.

[0099] The target material template includes at least one placeholder for the corresponding material type, which is used to fill in video material of that material type.

[0100] In one optional embodiment, generating a set of video materials related to the video category for each video instance identifier includes: obtaining a set of video material description information related to the video category for each video instance identifier, wherein each set of video material description information includes description information of multiple video materials; for each video instance identifier, based on the set of video material description information corresponding to the video instance identifier, calling multiple material generation models based on artificial intelligence to generate multiple video materials corresponding to the video instance identifier, and synchronously uploading the multiple video materials corresponding to the video instance identifier to the content delivery network; correspondingly, before performing video generation processing based on the target video templates and a set of video materials corresponding to N video instance identifiers to obtain N videos under the video category, the method further includes: in response to a batch video generation trigger event, obtaining multiple sets of video materials corresponding to the N video instance identifiers from the content delivery network according to the N video instance identifiers. Hosting video materials through the content delivery network reduces server-side storage pressure, enabling the server to efficiently render a large number of videos and improve user experience.

[0101] In this embodiment, the descriptive information of each group of video materials is used to generate video materials with corresponding video instance identifiers. Each group of video materials contains video materials of various types. Material type refers to the type of different video elements that constitute the video content, including but not limited to: audio, video, background images, subtitles, digital humans, etc.

[0102] In this embodiment, the implementation method of generating a set of video material description information related to the video category for each video instance identifier is not limited.

[0103] In one optional implementation, for any video instance identifier, description information of video materials can be randomly extracted from multiple material types stored in the basic material library, and the extracted description information of multiple video materials can be used as the description information of a group of video materials for any video instance identifier. The basic material library stores description information of multiple video materials under multiple video categories.

[0104] In another optional implementation, for any video instance identifier, the semantic similarity between the description information of multiple video materials in the basic material library and the video category is calculated to obtain multiple similarity information; the description information of the video materials that meet the similarity conditions among the multiple similarity information is used as the description information of a group of video materials for that video instance identifier.

[0105] In another optional implementation, for any video instance identifier, a material description information generation model is invoked based on the video category. This model is used to generate description information for various video materials related to that video category. Specifically, this material description information generation model is trained using description information from a large number of sample video materials across different video categories and material types. By learning the semantic relevance between the description information of different sample video categories and material types, the model can selectively combine different video categories to generate description information for various related video materials.

[0106] Furthermore, for each video instance identifier, based on a set of video material description information corresponding to that video instance identifier, multiple material generation models based on artificial intelligence are invoked to generate multiple video materials corresponding to that video instance identifier, and these multiple video materials corresponding to that video instance identifier are simultaneously uploaded to the content delivery network.

[0107] One material generation model can generate at least some of the video materials from multiple sources. For example, one material generation model can generate only one type of video material. The following explanation uses this as an example, but is not limited to it.

[0108] In this embodiment, when generating video materials with video instance identifiers, a target video template corresponding to each video instance identifier is determined based on the multiple video materials corresponding to each video instance identifier. The target video template corresponding to each video instance identifier is used to describe the rendering rules and hierarchical relationships of the multiple video materials corresponding to that video instance identifier. The method for determining the target video template corresponding to each video instance identifier can be referred to the above embodiment, and will not be repeated here.

[0109] In this embodiment, in response to the batch video generation trigger event, various video materials corresponding to the N video instance identifiers are obtained from the content delivery network according to the N video instance identifiers; videos are generated according to the various video materials corresponding to the N video instance identifiers and the target video template to obtain N videos under the video category.

[0110] In this embodiment, the specific implementation of the batch video generation trigger event is not limited and can be flexibly configured according to actual application needs. For example, it can be when all the various materials corresponding to N video instance identifiers have been generated; or, it can be when multiple materials corresponding to each video instance identifier are generated, in which case video generation will be performed on the various video materials of that video instance identifier; or, the batch video generation trigger event can also be a preset trigger time, such as a period of time after the input operation on the video generation page regarding the number of videos and video categories, for example, 2 hours, 1 day, or 1 week, etc., with no limitation on the time span.

[0111] It should be noted that each video instance identifier corresponds to the generation of one video, and N video instance identifiers can correspond to the generation of N videos. When generating videos for the various video materials corresponding to each of the N video instance identifiers, the generation process of the video corresponding to each video instance identifier is asynchronous, and the generation of videos corresponding to each video instance identifier does not affect each other, so as to improve the generation efficiency of N videos.

[0112] Further, optionally, the descriptive information for each set of video footage includes, but is not limited to: subtitle description information, audio type description information, digital human description information, and background image description information. For example... Figure 2As shown, based on a set of video material description information corresponding to the video instance identifier, multiple material generation models based on artificial intelligence are invoked to generate various video materials corresponding to the video instance identifier. This includes: generating text information using a generative language model based on subtitle description information to obtain target subtitles; converting target subtitles into target audio that matches the audio type description information using a text-to-speech model based on the target subtitles and audio type description information; selecting a target digital human based on the digital human description information using a multimodal model based on the target audio and the digital human description information, and generating a green screen video based on the target audio and the target digital human to obtain a green screen video of the target digital human; and generating a background image using a text-to-image model based on the background image description information to obtain a target background image.

[0113] In this embodiment, the APIs (Application Programming Interfaces) of multiple material generation models are associated with endpoints. In this embodiment, the video generation page is the presentation of the endpoint's front-end code. The endpoint is used for generating video materials, managing batch video generation tasks, and controlling the video generation process.

[0114] The endpoint includes front-end code and back-end code, adopting a client-server structure as described in the above embodiment. The front-end code refers to the video generation page built on the front-end framework. This page runs on the client side, interacts with the user, receives user input, and initiates batch video generation tasks to the server. The back-end code runs on the server side and is derived from the API of existing video editing software, such as Shortcut. In existing video editing software, the front-end UI code and video rendering function code are highly coupled. In this embodiment, the code is separated according to function to decouple the front-end UI and video rendering function code of the video editing software. The video rendering function code of the existing video editing software is then encapsulated into an independent API, allowing external code to call the video rendering function through a standard API, achieving automated video generation.

[0115] like Figure 2 In one example, the endpoint's front-end code could be a video generation page built on Astro, responsible for receiving callback notifications. For instance, when the material generation model is complete, it can send a notification to the endpoint to indicate that the subsequent video generation process can continue. The endpoint's back-end code could be obtained by API-izing the video rendering code of video editing software, exposing it externally for external calls via an API interface.

[0116] In one optional embodiment, the process of generating text information by calling a generative language model based on the subtitle description information to obtain target subtitles includes: matching corresponding keywords according to the video category, calling a pre-designed prompt word template, filling the prompt word template with keywords of the video category to obtain prompt words for the video category; inputting the prompt words of the video category into the generative language model to process the text information and obtain target subtitles related to the video category.

[0117] Keywords describe the theme of the video category; different video categories can correspond to different keywords, and different video types can also have different prompt word templates. Target subtitles include all the text content required for each video. By calling a generative language model, target subtitles can be automatically generated based on the video category without manual intervention, improving generation efficiency.

[0118] Optionally, based on the target subtitles and audio type description information, a text-to-speech model is invoked to convert the target subtitles into target audio that matches the audio type description information. The target subtitles provide the text content, and the audio type description information specifies the audio type of the generated target audio. Audio types include, but are not limited to, audio format, audio language, audio tone, and audio quality. Any audio type that can be used to specify the sound effects of the target audio is applicable to this embodiment. Based on the target subtitles and audio description information, the text-to-speech model is invoked to generate the corresponding target audio, which is then uploaded to a content delivery network, improving the automation of audio generation.

[0119] Furthermore, a multimodal model is invoked to generate a green screen video of a digital human. Here, "digital human" refers to a virtual character generated based on AI technology, capable of synchronously simulating the appearance, voice, lip movements, and other behaviors of a real person. It can be used in scenarios such as intelligent customer service, short video production, and virtual anchors, but is not limited to these. In this embodiment, by combining multiple video materials corresponding to each video instance identifier, a video with a realistic speaking effect can be generated.

[0120] In one optional embodiment, the digital human can be a pre-recorded video or image of a real person. In subsequent embodiments, the video and image are collectively referred to as video frames, and the number of video frames can be one or more. In this case, the descriptive information for different digital humans can be implemented as identification information. The identification information serves as a unique identifier for the digital human and is used to obtain the video frames of the digital human corresponding to the identification information. In another optional embodiment, the video frames of the digital human can be dynamically generated based on a multimodal model. In this case, the descriptive information of the digital human can be a prompt word used to generate the digital human, and the descriptive information of the digital human corresponding to each video instance can be different.

[0121] Optionally, when calling a multimodal model based on the target audio and digital human description information, selecting a target digital human based on the digital human description information, and generating a green screen video based on the target audio and the target digital human to obtain a green screen video of the target digital human, the process includes: obtaining the corresponding target digital human video frames based on the digital human description information; calling a multimodal model based on the target digital human video frames and the target audio to extract multidimensional features from the target audio to obtain multidimensional speech features; wherein, the multidimensional speech features include, but are not limited to: speech content features and speech emotion features; determining the lip-sync control parameters of the target digital human based on the speech content features; determining the facial expression control parameters of the target digital human based on the speech emotion features; determining the body movement control parameters of the target digital human based on the speech content features and the speech emotion features; generating a green screen video of the target digital human based on the lip-sync control parameters, facial expression control parameters, and body movement control parameters of the target digital human; and matching the lip-sync, facial expression, and body movement of the target digital human's green screen video with the target audio.

[0122] In this embodiment, speech content features are used to reflect the semantic information in the target audio, such as lexical content, grammatical structure, speech intent, and speech rhythm, and are mainly used to drive the lip movements of the digital human to be semantically synchronized with the target audio. Speech emotion features are used to reflect the emotional state of the target audio, including but not limited to tone of voice, speech rate, and intonation changes, and are mainly used to drive the facial expressions and body movements of the digital human.

[0123] Furthermore, based on the characteristics of speech content, the lip-shape control parameters of the target digital human are determined; based on the characteristics of speech emotion, the facial expression control parameters of the target digital human are determined; and based on the characteristics of speech content and speech emotion, the body movement control parameters of the target digital human are determined.

[0124] Having obtained lip-sync control parameters, facial expression control parameters, and body movement control parameters, the target digital human is driven to perform motion rendering based on these parameters, generating a corresponding green screen video of the target digital human. This allows for flexible replacement of the background image later. Since the green screen video of the target digital human is generated based on multi-dimensional speech features extracted from the target audio content, it ensures that the dynamic performance of the target digital human's lip-sync, facial expressions, and body movements in the green screen video matches the target audio semantically, temporally, and emotionally. This ensures precise lip-sync alignment and enhances the realism and viewing experience of the generated video.

[0125] Furthermore, in this embodiment, the target audio and target subtitles are aligned. For example, the target subtitles are time-stamped and aligned using the timestamps of the speech content in the target audio to ensure that the subsequent display of the target subtitles is completely synchronized with the target audio.

[0126] In this embodiment, one method for generating the background image is to call a text-based image model based on the background image description information to generate the target background image. Another method is to directly obtain a pre-generated or captured background image; this method is not limited.

[0127] In one optional embodiment, a set of video material generation states is maintained for each of the N video instance identifiers. The generation state of any video material of any type within each set includes: ready to generate, generating in progress, successfully generated, and failed generated. For example, for any video instance identifier, taking the multiple video materials that need to be generated for that video instance identifier, including: target subtitles, target audio, target digital human green screen video, and target background image, then the generation states for the target subtitles, target audio, target digital human green screen video, and target background image can be maintained separately.

[0128] Specifically, for each video instance identifier, the various video materials corresponding to that video instance identifier are marked as ready for generation. When calling multiple AI-based material generation models to generate the various video materials corresponding to that video instance identifier, if any material generation model returns a "generating in progress" response message, the video materials that the material generation model should generate are updated to the "generating in progress" state; if any material generation model returns a "generating successfully" message, the video materials that the material generation model should generate are updated to the "generating successfully" state; if any material generation model returns a "generating failed" message, the video materials that the material generation model should generate are updated to the "generating failed" state. For example... Figure 2 As shown, a subscription service is provided that can notify each material generation model of the generation status of the video material for that model, such as a successful generation status. The subscription service then returns a successful generation message to the server to inform it that the corresponding video material has been successfully generated. Figure 2 This example only uses the notification process for the target digital person as an example, but is not limited to this.

[0129] Optionally, if the generation status of the video footage is updated to a generation failure status, a failure notification message is output to the user who initiated the input operation. The failure notification message includes the footage type of the video footage that has been updated to a generation failure status and its corresponding video instance identifier. If a regeneration operation triggered by the user is received, the corresponding description information of the video footage is obtained based on the video instance identifier, and the corresponding AI-based footage generation model is invoked to regenerate the corresponding video footage.

[0130] In this embodiment, when multiple video materials corresponding to each video instance identifier are generated, these multiple video materials can be simultaneously uploaded to the content delivery network (CDN), and access links for the video materials corresponding to each video instance identifier can be obtained within the CDN. Continuing with the example above, for any video instance identifier, if the multiple video materials corresponding to that video instance identifier include target subtitles, target audio, target digital human green screen video, and target background image, the target subtitles, target audio, target digital human green screen video, and target background image are uploaded to the CDN respectively, and access links for the target subtitles, target audio, target digital human green screen video, and target background image are obtained.

[0131] Furthermore, in response to the batch video generation trigger event, based on the access links of the various video materials corresponding to each of the N video instance identifiers on the content delivery network, the various video materials corresponding to the N video instance identifiers are obtained respectively; and, based on the various video materials corresponding to each of the N video instance identifiers and the target video template, videos are generated to obtain N videos under the video category.

[0132] Following the example above, in response to the batch video generation trigger event, based on the access links of the target subtitles, target audio, target digital human green screen video, and target background image corresponding to each of the N video instance identifiers in the content delivery network, the target subtitles, target audio, target digital human green screen video, and target background image corresponding to each of the N video instance identifiers are obtained respectively; and, based on the target subtitles, target audio, target digital human green screen video, target background image, and target video template corresponding to each of the N video instance identifiers, videos are generated to obtain N videos under the video category.

[0133] Optionally, the videos and target video templates corresponding to the N video instance identifiers are uploaded to the content delivery network, and access links for the videos and target video templates corresponding to the N video instance identifiers in the content delivery network are obtained. The access links are added to the video generation results page. In response to the viewing operation of the video results page, the video generation results page is displayed. The video generation results page contains a set of video materials, videos, and access links for the target video templates corresponding to at least one of the N video instance identifiers in the content delivery network.

[0134] In this embodiment, each video instance identifies its own video materials, video, and target video template, which are stored on the content delivery network. This eliminates the need for the server to store a large number of files, reducing disk I / O load and ensuring service stability. Furthermore, storing access links for each video instance's identified video materials, video, and target video template on the content delivery network avoids data bloat, reduces query pressure, and supports larger-scale data management.

[0135] In this optional embodiment, in response to a triggered operation of an access link in the content delivery network corresponding to at least one video instance identifier, the set of video materials, videos, and / or target video templates corresponding to at least one video instance identifier is accessed. By hosting videos and video templates through a CDN, the video generation result page can directly load videos from the content delivery network. Compared to pulling videos from the server, this consumes less bandwidth, loads faster, and significantly improves performance, making it suitable for large-scale video browsing scenarios.

[0136] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0137] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 103 can be device A; or the execution subject of steps 101 and 402 can be device A, and the execution subject of step 103 can be device B; and so on.

[0138] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0139] Figure 3 This is a schematic diagram of the structure of an electronic device provided as another exemplary embodiment of this application. For example... Figure 3 As shown, the device includes a memory 34 and a processor 35.

[0140] Memory 34 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, data structures, contact data, phone book data, messages, pictures, videos, etc.

[0141] Processor 35, coupled to memory 34, is configured to execute a computer program in memory 34 for: responding to input operations on a video generation page regarding the number of videos and video categories; generating a batch video generation task, the batch video generation task including the number of videos N and video categories, where N is an integer ≥ 2; generating N video instance identifiers based on the batch video generation task, and generating a set of video materials related to the video category for each video instance identifier; and obtaining multiple material templates obtained by variable processing of an initial video template, the initial video template including rendering rules for multiple video materials required to generate videos and the hierarchical relationship between the multiple video materials, each material... A rendering template is used to describe the rendering rules of a video material. For each video instance identifier, at least one target rendering template is determined from the multiple rendering templates according to the material type in the set of video materials corresponding to the video instance identifier. Based on the hierarchical relationship between the multiple video materials included in the initial video template, the at least one target rendering template is combined to obtain the target video template corresponding to the video instance identifier. The target video template is used to describe the rendering rules and hierarchical relationship of the set of video materials. Video generation processing is performed according to the target video templates corresponding to N video instance identifiers and a set of video materials to obtain N videos under the video category.

[0142] In an optional embodiment, when the processor 35 performs variable processing on the initial video template, it is specifically used to: parse the initial video template with the material type as the splitting variable to obtain information fragments corresponding to multiple material types; extract rendering parameter information corresponding to each of the multiple material types from the information fragments corresponding to each of the multiple material types; and template the rendering parameter information corresponding to each of the multiple material types to obtain the multiple material templates.

[0143] In an optional embodiment, when the processor 35 determines at least one target material template from the plurality of material templates based on the material type in a set of video materials corresponding to the video instance identifier, it is specifically configured to: identify at least one material type contained in the set of video materials corresponding to the video instance identifier; select at least one initial material template from the plurality of material templates based on the at least one material type contained in the set of video materials, wherein each material type corresponds to one initial material template; and adjust the parameters of at least some of the initial material templates in the at least one initial material template to obtain the at least one target material template.

[0144] In an optional embodiment, the initial material template includes at least one variable parameter, and each variable parameter is associated with multiple candidate parameter values ​​and a default parameter value; then, when the processor 35 adjusts the parameters of at least a portion of the at least one initial material template to obtain the at least one target material template, it specifically performs the following: determining the number of templates whose parameters need to be adjusted, wherein the number of templates is less than or equal to the number of the at least one initial material template; selecting initial material templates to be adjusted from the at least one initial material template according to the number of templates; determining the variable parameter to be adjusted from the initial material templates to be adjusted; randomly determining a target parameter value from the multiple candidate parameter values ​​associated with the variable parameter to be adjusted; and assigning the target parameter value to the variable parameter to be adjusted to obtain the target material template.

[0145] In an optional embodiment, when the processor 35 combines the at least one target material template based on the hierarchical relationship between the various video materials included in the initial video template to obtain the target video template corresponding to the video instance identifier, it is specifically configured to: generate a basic video template according to the hierarchical relationship between the various video materials included in the initial video template, the basic video template including multiple blank structural positions corresponding to the various video materials; the positional relationship between the multiple blank structural positions reflects the hierarchical relationship between the multiple video materials; insert the at least one target material template into the corresponding blank structural positions in the basic video template to obtain the target video template corresponding to the video instance identifier; or, according to the at least one target material template, cover the structural positions where the rendering rules of video materials of the same material type in the initial video template are located to obtain the target video template corresponding to the video instance identifier; wherein, in the initial video template, the rendering rules of each type of video material occupy one structural position, and the positional relationship between the structural positions reflects the hierarchical relationship between the multiple video materials.

[0146] In an optional embodiment, when the processor 35 performs video generation processing based on the target video templates and a set of video materials corresponding to each of the N video instance identifiers to obtain N videos under the video category, it specifically performs the following steps: for each video instance identifier, it fills the target material template in the target video template corresponding to the video instance identifier into the target material template; for the filled target video template, it renders the set of video materials according to the rendering rules and hierarchical relationships of the set of video materials described in the filled target video template to obtain a video under the video category.

[0147] In an optional embodiment, when the processor 35 generates a set of video materials related to the video category for each video instance identifier, it is specifically configured to: obtain a set of video material description information related to the video category for each video instance identifier, wherein each set of video material description information includes description information of multiple video materials; for each video instance identifier, according to the set of video material description information corresponding to the video instance identifier, call multiple material generation models based on artificial intelligence to generate multiple video materials corresponding to the video instance identifier, and synchronously upload the multiple video materials corresponding to the video instance identifier to the content distribution network; correspondingly, before the processor 35 performs video generation processing based on the target video templates and a set of video materials corresponding to each of the N video instance identifiers to obtain N videos under the video category, it is further configured to: in response to a batch video generation trigger event, obtain multiple sets of video materials corresponding to the N video instance identifiers from the content distribution network according to the N video instance identifiers respectively.

[0148] In an optional embodiment, the description information of each set of video materials includes: subtitle description information, audio type description information, digital human description information, and background image description information. When the processor 35 generates multiple video materials corresponding to the video instance identifier by calling multiple AI-based material generation models based on the description information of a set of video materials corresponding to the video instance identifier, it specifically performs the following: Based on the subtitle description information, it calls a generative language model to generate text information to obtain target subtitles; based on the target subtitles and the audio type description information, it calls a text-to-speech model to convert the target subtitles into target audio that matches the audio type description information; based on the target audio and the digital human description information, it calls a multimodal model to select a target digital human based on the digital human description information, and generates a green screen video based on the target audio and the target digital human to obtain the green screen video of the target digital human; based on the background image description information, it calls a text-to-image model to generate a background image to obtain a target background image.

[0149] In an optional embodiment, when the processor 35 invokes a multimodal model based on the target audio and the digital human description information, selects a target digital human based on the digital human description information, and generates a green screen video based on the target audio and the target digital human to obtain a green screen video of the digital human, the processor 35 specifically performs the following steps: Based on the digital human description information, it obtains the corresponding target digital human video frame; based on the target digital human video frame and the target audio, it invokes a multimodal model to extract multidimensional features from the target audio to obtain multidimensional speech features, the multidimensional speech features including: speech content features and speech emotion features; based on the speech content features, it determines the lip-sync control parameters of the target digital human; based on the speech emotion features, it determines the facial expression control parameters of the target digital human; based on the speech content features and speech emotion features, it determines the body movement control parameters of the target digital human; based on the lip-sync control parameters, expression control parameters, and body movement control parameters of the target digital human, it generates a green screen video of the target digital human; the lip-sync, facial expression, and body movement of the target digital human's green screen video are matched with the target audio.

[0150] Furthermore, such as Figure 3 As shown, the electronic device also includes other components such as a communication component 36, a display 37, a power supply component 38, and an audio component 39. Figure 3 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 3 The components shown. Additionally... Figure 3 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product form of the electronic device. The electronic device in this embodiment can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or a server-side device such as a conventional server, cloud server, or server array. If the electronic device in this embodiment is a terminal device such as a desktop computer, laptop computer, or smartphone, it may include... Figure 3 The components within the dashed box; if the electronic device in this embodiment is implemented as a conventional server, cloud server, or server array, etc., it may be omitted. Figure 3 The component within the dashed box.

[0151] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0152] The aforementioned communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0153] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0154] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0155] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0156] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile components, or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium.

[0157] Accordingly, this application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above method embodiments.

[0158] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0159] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for batch video generation, characterized in that, include: In response to input operations on the video generation page regarding the number of videos and video categories, a batch video generation task is generated. The batch video generation task includes the number of videos N and the video categories, where N is an integer ≥ 2. Based on the batch video generation task, N video instance identifiers are generated, and a set of video materials related to the video category are generated for each video instance identifier; Obtaining multiple material templates by variableizing an initial video template includes: parsing the initial video template using material type as a splitting variable to obtain information fragments corresponding to multiple material types; extracting rendering parameter information corresponding to each of the multiple material types from the information fragments corresponding to each of the multiple material types; converting the rendering parameter information corresponding to each of the multiple material types from rendering parameters with fixed configuration default parameter values ​​into a set of variable parameters with dynamically replaceable parameter values ​​to obtain the multiple material templates; the initial video template includes rendering rules for multiple video materials required to generate the video and the hierarchical relationship between the multiple video materials, with each material template describing the rendering rules for one type of video material; For each video instance identifier, based on the material type in the set of video materials corresponding to the video instance identifier, at least one target material template is determined from the multiple material templates. The target material template is obtained by adjusting the parameters of at least some of the initial material templates selected from the multiple material templates. Based on the hierarchical relationship between the multiple video materials included in the initial video template, the at least one target material template is combined to obtain the target video template corresponding to the video instance identifier. The target video template is used to describe the rendering rules and hierarchical relationship of the set of video materials. Based on the target video templates and a set of video materials corresponding to each of the N video instance identifiers, video generation processing is performed to obtain N videos under the video category.

2. The method according to claim 1, characterized in that, Based on the material type in a set of video materials corresponding to the video instance identifier, at least one target material template is determined from the multiple material templates, including: Identify at least one type of material contained in a group of video materials corresponding to the video instance identifier; Based on at least one material type included in the set of video materials, at least one initial material template is selected from the multiple material templates, with each material type corresponding to one initial material template; At least a portion of the initial material templates are adjusted in terms of parameters to obtain the at least one target material template.

3. The method according to claim 2, characterized in that, The initial material template includes at least one variable parameter, and each variable parameter is associated with multiple candidate parameter values ​​and default parameter values; then, at least a portion of the initial material templates are adjusted to obtain the at least one target material template, including: The number of templates whose parameters are to be adjusted is determined, wherein the number of templates is less than or equal to the number of the at least one initial material template; Based on the number of templates, select the initial material template to be adjusted from the at least one initial material template; Determine the variable parameters to be adjusted from the initial material template to be adjusted; The target parameter value is randomly determined from multiple candidate parameter values ​​associated with the variable parameter to be adjusted; The target parameter value is assigned to the variable parameter to be adjusted to obtain the target material template.

4. The method according to claim 1, characterized in that, Based on the hierarchical relationship between the various video materials included in the initial video template, the at least one target material template is combined to obtain the target video template corresponding to the video instance identifier, including: Based on the hierarchical relationship between the various video materials included in the initial video template, a basic video template is generated. The basic video template includes multiple blank structural positions corresponding to the various video materials. The positional relationship between the multiple blank structural positions reflects the hierarchical relationship between the multiple video materials. The at least one target material template is inserted into the corresponding blank structural position in the basic video template to obtain the target video template corresponding to the video instance identifier. or Based on the at least one target material template, the structural positions where the rendering rules of video materials of the same material type in the initial video template are located are overwritten to obtain the target video template corresponding to the video instance identifier; wherein, in the initial video template, the rendering rules of each type of video material occupy one structural position, and the positional relationship between the structural positions reflects the hierarchical relationship between the multiple video materials.

5. The method according to claim 1, characterized in that, Based on the target video templates and a set of video materials corresponding to N video instance identifiers, video generation processing is performed to obtain N videos under the aforementioned video category, including: For each video instance identifier, a set of video materials corresponding to the video instance identifier are respectively filled into the target material template in the target video template corresponding to the video instance identifier; For the filled target video template, the set of video materials are rendered according to the rendering rules and hierarchical relationships described in the filled target video template to obtain a video under the video category.

6. The method according to claim 1, characterized in that, For each video instance, a set of video materials related to the video category is generated, including: For each video instance identifier, obtain a set of video material description information related to the video category. Each set of video material description information includes description information of multiple video materials. For each video instance identifier, based on a set of video material description information corresponding to the video instance identifier, multiple material generation models based on artificial intelligence are invoked to generate multiple video materials corresponding to the video instance identifier, and the multiple video materials corresponding to the video instance identifier are synchronously uploaded to the content delivery network. Accordingly, before performing video generation processing based on the target video templates corresponding to each of the N video instance identifiers and a set of video materials to obtain the N videos under the video category, the process further includes: In response to the batch video generation trigger event, multiple sets of video materials corresponding to the N video instance identifiers are obtained from the content delivery network according to the N video instance identifiers.

7. The method according to claim 5, characterized in that, The description information for each set of video materials includes: subtitle description information, audio type description information, digital human description information, and background image description information. Based on the description information of the set of video materials corresponding to the video instance identifier, multiple AI-based material generation models are invoked to generate various video materials corresponding to the video instance identifier, including: Based on the subtitle description information, a generative language model is invoked to generate text information to obtain the target subtitle; Based on the target subtitle and the audio type description information, a text-to-speech model is invoked to convert the target subtitle into target audio that matches the audio type description information; Based on the target audio and the digital human description information, a multimodal model is invoked, a target digital human is selected based on the digital human description information, and a green screen video is generated based on the target audio and the target digital human to obtain the green screen video of the target digital human. Based on the background image description information, the text image model is invoked to generate the background image to obtain the target background image.

8. The method according to claim 7, characterized in that, Based on the target audio and the digital human description information, a multimodal model is invoked to select a target digital human according to the digital human description information, and a green screen video is generated based on the target audio and the target digital human to obtain a digital human green screen video, including: Based on the digital human description information, obtain the corresponding target digital human video frame; Based on the video frames and audio of the target digital human, a multimodal model is invoked to extract multidimensional features from the target audio to obtain multidimensional speech features, which include: speech content features and speech emotion features. Based on the speech content features, lip-sync control parameters of the target digital human are determined; based on the speech emotion features, facial expression control parameters of the target digital human are determined; and based on the speech content features and speech emotion features, body movement control parameters of the target digital human are determined. Based on the lip-sync control parameters, facial expression control parameters, and body movement control parameters of the target digital human, a green screen video of the target digital human is generated; the lip-sync, facial expression, and body movement of the target digital human in the green screen video are matched with the target audio.

9. An electronic device, characterized in that, The method includes a memory and a processor, the memory being used to store a computer program, and the processor being coupled to the memory for executing the computer program to implement the steps of the method according to any one of claims 1-8.

10. A computer-readable storage medium storing a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the processor is enabled to perform the steps of the method according to any one of claims 1-8.

11. A computer program product, characterized in that, include: A computer program / instruction that, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1-8.

Images