Distributed video generation method, device, storage medium, and program product

Abstract

The disclosed embodiments disclose a distributed video generation method, device, storage medium, and program product. The method includes: obtaining an original text, splitting the original text into multiple sub-texts; setting multiple execution nodes, and setting priorities for the multiple execution nodes; traversing the multiple sub-texts according to the order of the multiple sub-texts in the original text to determine the current sub-text; assigning the current sub-text to the execution node with the highest current priority to generate a video segment; synthesizing all generated video segments into a target video. The method can reduce the time consumption and greatly improve the efficiency of video generation while ensuring the quality of video generation through efficient resource allocation and processing strategies.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a distributed video generation method, device, storage medium, and program product. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the continuous progress of general artificial intelligence (AGI), AI-generated video technology, as an emerging video production method, has shown great potential and application value in many fields such as media, entertainment, and education. AI-generated video technology greatly improves the efficiency of video production and reduces costs by automatically converting text descriptions into video content, providing unprecedented convenience for content creators. However, the video generation process is usually time-consuming and inefficient. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide a distributed video generation method, device, storage medium, and program product, which can reduce time consumption and significantly improve video generation efficiency while ensuring video generation quality through efficient resource allocation and processing strategies.

[0004] In a first aspect, the embodiments of the present disclosure provide a distributed video generation method, which adopts the following technical solutions:

[0005] Obtaining an original text, and splitting the original text into multiple sub-texts;

[0006] Setting multiple execution nodes and setting priorities for the multiple execution nodes;

[0007] According to the sequential positions of the multiple subtexts in the original text, traverse the multiple subtexts to determine the current subtext;

[0008] Allocating the current subtext to the execution node with the highest current priority to generate a video clip;

[0009] All generated video segments are synthesized into the target video.

[0010] Optionally, splitting the original text into multiple sub-texts includes:

[0011] Splitting the original text into multiple text segments according to preset symbols;

[0012] Sort the multiple text fragments according to their positions in the original text to obtain a total number of text fragments;

[0013] Determine whether the number of characters in the i-th text segment is greater than a preset character threshold;

[0014] If the number of characters in the i-th text segment is greater than a preset character threshold, or the total number of characters in the i-th text segment and the i+1-th text segment is greater than a preset character threshold, the i-th text segment is taken as a subtext, i=i+1;

[0015] If the total number of characters of the i-th text segment and the i+1-th text segment is not greater than the preset character threshold, the i-th text segment and the i+1-th text segment are merged to generate a new i-th text segment, and whether the total number of characters of the new i-th text segment and the i+1-th text segment is not greater than the preset character threshold is continued to be determined; the multiple text segments are reordered, and the total number of text segments is updated;

[0016] When i is greater than the total number of text segments, it indicates that the original text has been split.

[0017] Optionally, assigning priorities to the multiple execution nodes includes:

[0018] Each execution node includes a plurality of sub-nodes, and the sub-nodes are used to generate a video segment according to the assigned current sub-text;

[0019] Real-time statistics of the number of idle child nodes of each execution node;

[0020] sorting the plurality of execution nodes according to the number of idle child nodes;

[0021] The sorting result is used as the priority of the multiple execution nodes.

[0022] Optionally, setting priorities for the multiple execution nodes includes:

[0023] Each execution node includes a plurality of sub-nodes, and the sub-nodes are used to generate a video segment according to the assigned current sub-text;

[0024] Real-time statistics of the number of idle child nodes of each execution node;

[0025] Based on the number of idle child nodes, obtaining scores of the plurality of execution nodes;

[0026] sorting the plurality of execution nodes according to the scores;

[0027] Using the sorting result as the priority of the multiple execution nodes;

[0028] The calculation formula of the score is as follows:

[0029] ;

[0030] Where x is the sequence number of the execution node; j is the sequence number of the historical subtext, and J is the total number of historical subtexts; P x is the score of the xth execution node; w1, w2, and w3 are all preset weight values; F x The number of idle child nodes of the xth execution node; is the execution efficiency of the xth execution node on the jth historical subtext, ;t j Z is the time taken by the xth execution node to generate a video segment based on the jth historical subtext; j is the number of characters in the jth historical subtext; G x is the failure frequency of the xth execution node.

[0031] Optionally, the distributed video generation method further includes:

[0032] Each execution node includes a plurality of sub-nodes, and the sub-nodes are used to generate a video segment according to the assigned current sub-text;

[0033] Real-time statistics of the number of idle child nodes of each execution node;

[0034] When the execution node is stored in the priority queue and the number of idle child nodes of the execution node is 0, the execution node is moved from the priority queue to the full-load queue without setting the priority of the execution node in the full-load queue;

[0035] When the execution node is stored in the full-load queue and the number of idle child nodes of the execution node is greater than a preset threshold, or all child nodes of the execution node are idle, the execution node is moved from the full-load queue to the priority queue.

[0036] Optionally, the distributed video generation method further includes:

[0037] When the execution node fails, the execution node is removed from the original queue, where the original queue is the priority queue or the full-load queue;

[0038] Re-prioritize non-faulty execution nodes;

[0039] Move all subtexts out of the failed execution node;

[0040] Reassign the removed subtexts according to the new priority level;

[0041] After the faulty execution node is modified, the modified execution node is stored in the priority queue again.

[0042] Optionally, the distributed video generation method further includes:

[0043] When a user provides an original video, the current subtext is added as a subtitle to the original video to generate a subtitle video, and a segment with subtitles is extracted from the subtitle video as a video segment;

[0044] When the user does not provide the original video, a large language model is used to convert the current subtext into the video clip, or a video material matching the current subtext is selected from a material library, subtitles are configured for the video material, and the video clip is generated.

[0045] In a second aspect, the embodiment of the present disclosure further provides a distributed video generation system, which adopts the following technical solution:

[0046] A splitting module, used for acquiring an original text and splitting the original text into multiple sub-texts;

[0047] A setting module, used to set multiple execution nodes and set priorities for the multiple execution nodes;

[0048] A traversal module, used for traversing the multiple subtexts according to the sequential positions of the multiple subtexts in the original text to determine the current subtext;

[0049] An allocation module, used for allocating the current subtext to the execution node with the highest current priority to generate a video segment;

[0050] The synthesis module is used to synthesize all generated video clips into the target video.

[0051] In a third aspect, the embodiments of the present disclosure further provide a computer device, which adopts the following technical solution:

[0052] The computer device comprises:

[0053] at least one processor; and,

[0054] a memory communicatively connected to the at least one processor; wherein,

[0055] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform any of the above-mentioned distributed video generation methods.

[0056] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned distributed video generation methods.

[0057] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program / instruction, which implements the steps of any of the above methods when executed by a processor.

[0058] The distributed video generation method provided by the embodiment of the present disclosure can process multiple sub-texts in parallel by splitting the original text into multiple sub-texts and assigning them to different execution nodes, thereby significantly improving the overall efficiency of video generation. Each node can generate video clips independently, reducing the burden of a single node and shortening the generation time. Setting the priority of the execution node can ensure the optimal allocation of computing resources, maximize the resource utilization of the system, and avoid idle or overcrowded resources. By distributing the generation process to multiple execution nodes, each execution node can focus on the part it is responsible for, ensuring that each video clip can meet a high quality standard. In the distributed generation process, if an execution node fails, other nodes can still continue to work. This method can improve the robustness and stability of the system and reduce the risk of overall generation task failure due to single point failure. The number and priority of the execution nodes can also be dynamically adjusted according to demand, so as to flexibly respond to video generation tasks of different scales and complexities. This scalability enables the system to save resources when processing small-scale tasks and make full use of resources when processing large-scale tasks. The various video clips generated by distributed processing can be optimized through a unified synthesis strategy, so that the final target video is more logically coherent and more visually consistent. Through this efficient processing and generation, we can quickly respond to user needs, provide faster services, better meet user needs for video content generation speed and quality, and improve user satisfaction.

[0059] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0061] Figure 1 A schematic diagram of a process flow of a distributed video generation method provided by an embodiment of the present disclosure;

[0062] Figure 2A flowchart of a text splitting method provided in an embodiment of the present disclosure;

[0063] Figure 3 A flowchart of a priority setting method provided in an embodiment of the present disclosure;

[0064] Figure 4 Another schematic diagram of a priority setting method provided in an embodiment of the present disclosure;

[0065] Figure 5 A schematic diagram of a video generation process provided by an embodiment of the present disclosure;

[0066] Figure 6 A block diagram of the principle of a distributed video generation system provided by an embodiment of the present disclosure;

[0067] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0068] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0069] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0070] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0071] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0072] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0073] Reference Figure 1 The present disclosure provides a distributed video generation method, comprising the following steps:

[0074] S1: obtain the original text and split the original text into multiple sub-texts;

[0075] S2: Set multiple execution nodes and set priorities for the multiple execution nodes;

[0076] S3: traverse the multiple subtexts according to their positions in the original text to determine the current subtext;

[0077] S4: Assign the current subtext to the execution node with the highest current priority to generate a video clip;

[0078] S5: All generated video clips are synthesized into the target video.

[0079] The distributed video generation method disclosed in the present invention can process multiple sub-texts in parallel by splitting the original text into multiple sub-texts and assigning them to different execution nodes, thereby significantly improving the overall efficiency of video generation. Each node can generate video clips independently, reducing the burden of a single node and shortening the generation time. Setting the priority of the execution node can ensure the optimal allocation of computing resources, maximize the resource utilization of the system, and avoid idle or overcrowded resources. By distributing the generation process to multiple execution nodes, each execution node can focus on the part it is responsible for, ensuring that each video clip can meet a high quality standard. In the distributed generation process, if an execution node fails, other nodes can still continue to work. This method can improve the robustness and stability of the system and reduce the risk of overall generation task failure due to single point failure. The number and priority of the execution nodes can also be dynamically adjusted according to demand, so as to flexibly respond to video generation tasks of different scales and complexities. This scalability enables the system to save resources when processing small-scale tasks and make full use of resources when processing large-scale tasks. The various video clips generated by distributed processing can be optimized through a unified synthesis strategy, so that the final target video is more logically coherent and more visually consistent. Through this efficient processing and generation, we can quickly respond to user needs, provide faster services, better meet user needs for video content generation speed and quality, and improve user satisfaction.

[0080] In summary, this distributed video generation method can greatly improve generation efficiency and system stability while ensuring video generation quality through efficient resource allocation and processing strategies.

[0081] In S1, one possible implementation method of "splitting the original text into multiple sub-texts" is: using a semantic model to perform semantic splitting on the original text to obtain multiple sub-texts, and controlling the number of characters of each sub-text not to exceed a preset character threshold. Specifically, the semantic model uses natural language processing technology to perform grammatical and semantic analysis on the original text to identify keywords, phrases, and concepts in the text, and ensures that each sub-text is semantically complete and independent by identifying semantic units in the text, such as independent sentences or paragraphs. During the splitting process, the length of the sub-text is monitored in real time to ensure that it does not exceed the preset character threshold, so as to ensure that the length of each sub-text processed by the subsequent execution node is not too long, so as to improve the overall execution efficiency.

[0082] Reference Figure 2 The flowchart of the text splitting method shown in the figure shows that another possible implementation method of "splitting the original text into multiple sub-texts" includes the following steps:

[0083] S11: splitting the original text into multiple text segments according to preset symbols;

[0084] S12: sorting the multiple text fragments according to their positions in the original text to obtain the total number of text fragments;

[0085] S13: Determine whether the number of characters in the i-th text segment is greater than a preset character threshold; if so, execute S14; if not, execute S15;

[0086] S14: take the i-th text segment as a subtext, i=i+1, and execute S17;

[0087] S15: Determine whether the total number of characters in the i-th text segment and the i+1-th text segment is greater than a preset character threshold; if so, execute S14; if not, execute S16;

[0088] S16: merge the i-th text segment and the i+1-th text segment to generate a new i-th text segment, re-sort the multiple text segments, update the total number of text segments, and return to S15;

[0089] S17: Determine whether i is greater than the total number of text segments; if so, execute S18; if not, return to S13;

[0090] S18: Determine whether the original text is split completely.

[0091] The original text can be an existing text uploaded by the user, or it can be generated by LLM, that is, a large language model (LLM) is used to generate a complete original text based on the text content uploaded by the user. The preset symbol is a period, and the preset character threshold is 500. The original text is cleaned to remove invalid and erroneous data to ensure the quality of the text. Then, the cleaned original text is split according to the period. The generated text fragments will not exceed 500 characters. Even if they exceed 500 characters, they will hardly exceed much. By sorting multiple text fragments and re-sorting them after merging adjacent text fragments, the foundation is laid for the subsequent execution of the sequential process according to the size of i, where the initial value of i is 1. By merging text fragments with too small a number of characters into sub-texts, the number of characters of all sub-texts is controlled to be close to the preset character threshold, so that the workload and time of each execution node processing a sub-text tend to be consistent, so as to reduce the overall target video generation time and improve the overall target video generation efficiency.

[0092] Reference Figure 3 A flowchart of a priority setting method is shown, in S2, one possible implementation method of "setting priorities for multiple execution nodes" includes the following steps:

[0093] S201: Each execution node includes multiple child nodes;

[0094] S202: Count the number of idle child nodes of each execution node in real time;

[0095] S203: sorting the multiple execution nodes according to the number of idle child nodes;

[0096] S204: Using the sorting results as priorities of the multiple execution nodes.

[0097] In S201 , the sub-node is used to generate a video segment according to the assigned current sub-text.

[0098] In S202, when a subnode is not assigned any subtext, it is in an idle state; when a current subtext is assigned to a subnode, the subnode is in an occupied state; when the subnode has generated a video clip according to the assigned current subtext and the resource is released, the subnode is in an idle state again. According to this principle, the number of idle subnodes of each execution node is counted in real time.

[0099] In S203 and S204, all execution nodes are sorted in descending order according to the number of idle child nodes, and the priorities of all execution nodes are set according to the sorting results. For example, there are 5 execution nodes, namely A1, A2, A3, A4, and A5. The number of idle child nodes of the 5 execution nodes is counted and sorted according to the number of idle child nodes. The sorting result is A1→A4→A3→A5→A2, and the priorities of the 5 execution nodes are also reduced in descending order according to the sorting result.

[0100] Reference Figure 4 Another flowchart of the priority setting method is shown, and another possible implementation method of "setting priorities for multiple execution nodes" includes the following steps:

[0101] S211: Each execution node includes multiple child nodes;

[0102] S212: Count the number of idle child nodes of each execution node in real time;

[0103] S213: Obtaining scores of multiple execution nodes based on the number of idle child nodes;

[0104] S214: sorting the multiple execution nodes according to the scores;

[0105] S215: Using the sorting results as priorities of the multiple execution nodes.

[0106] Among them, the operating principles of S211 and S212 are the same as those of S201 and S202, and will not be repeated here.

[0107] In S213, the score calculation formula is as follows:

[0108] ;

[0109] ;

[0110] Where x is the sequence number of the execution node; j is the sequence number of the historical subtext, and J is the total number of historical subtexts; P x is the score of the xth execution node; w1, w2, and w3 are all preset weight values; F x The number of idle child nodes of the xth execution node; is the average execution efficiency of the x-th pair of execution nodes on the historical subtext; is the execution efficiency of the xth execution node on the jth historical subtext; G x is the failure frequency of the xth execution node; t j Z is the time taken by the xth execution node to generate a video segment based on the jth historical subtext; j is the number of characters in the jth historical subtext.

[0111] In S214 and S215, all execution nodes are sorted in descending order according to the scores, and the priorities of all execution nodes are set according to the sorting results. For example, there are 5 execution nodes, namely B1, B2, B3, B4, and B5. The scores of the 5 execution nodes are calculated and sorted according to the scores. The sorting result is B4→B5→B2→B3→B1, and the priorities of the 5 execution nodes are also reduced in descending order according to the sorting result.

[0112] Optionally, for ease of management, each execution node contains the same number of sub-nodes, and the hardware resources (such as CPU, GPU, memory, etc.) of each execution node or each sub-node can also be set to be the same.

[0113] The above method of setting priorities comprehensively considers the number of idle child nodes of the execution node, the average execution efficiency of the historical subtexts, and the failure frequency. By considering the number of idle child nodes, it is possible to ensure that the video clip generation task is assigned to the part that currently has processing capabilities, thereby improving the overall resource utilization. By considering the average execution efficiency of the historical subtexts, the video clip generation task can be preferentially assigned to the execution nodes with high historical execution efficiency, reducing the average time required to complete the video clip generation task, and speeding up the entire target video generation process. By considering the failure frequency, the risk of delay or failure of the video clip generation task due to execution node failure can be reduced, and the stability and reliability of the system can be improved.

[0114] In S3 and S4, each sub-text is traversed according to the sorting order of each sub-text in S2, that is, according to the order of each sub-text in the original text, and the sub-text being traversed is identified as the current sub-text. The current sub-text is assigned to the execution node with the highest priority, so that the video clip can be generated at the fastest speed.

[0115] Furthermore, a priority queue and a full-load queue are preset, and the priority queue and the full-load queue together constitute the ETCD (Etcd Distributed Key-Value Store) global queue, wherein the priority queue is implemented using a large heap data structure to maintain a stable order of the priority queue, ensure that the tasks and requests of the priority queue are responded to quickly, and facilitate the functional integration of the priority queue. In the initial stage, the priority queue stores all execution nodes. If it is detected that the number of idle child nodes of the execution node is 0, the execution node is moved from the priority queue to the full-load queue. When the number of idle child nodes of the execution node is greater than the preset threshold, or all child nodes of the execution node are idle, the execution node is moved from the full-load queue to the priority queue. The preset threshold is a value greater than or equal to 0.

[0116] The priority settings mentioned above are only for the execution nodes in the priority queue, that is, the priority is only set for the execution nodes in the priority queue, and the priority is not set for the execution nodes in the full-load queue. Through the cooperation of the priority queue and the full-load queue, invalid priority calculations for the full-load nodes are avoided, computing resources are saved, and priority calculations are ensured to focus on execution nodes with idle capacity, speeding up task allocation. By distinguishing the two queues, more effective load balancing is achieved to prevent overload, and idle execution nodes can be quickly identified and utilized, shortening task waiting time and improving system responsiveness.

[0117] Furthermore, when an execution node is detected to have a failure (due to network instability or downtime, etc.), the execution node is removed from the original queue (priority queue or full-load queue), all sub-texts in the execution node are removed, the priorities of the remaining execution nodes are updated, and these sub-texts are reallocated according to the new priorities. The status of the execution node is updated in the registration center, marked as unavailable, and the system administrator is notified of the failure event, and the fault is diagnosed and repaired. Before the execution node is repaired and put back online, the preset text is used to evaluate whether it has the ability to execute again. If it has the ability to execute again, the status of the execution node is updated in the registration center as "available", and the execution node is moved back to the priority queue.

[0118] When the execution node performs the video clip generation task, an automatic retry mechanism is adopted. When the video clip generation task fails, it automatically tries to execute it again. If the number of consecutive failures exceeds the preset threshold, the execution node is regarded as faulty.

[0119] By adopting the above-mentioned management method for execution nodes, it is possible to achieve a reasonable allocation of video clip generation tasks, improve the system's ability to process tasks, and increase the number of tasks completed per unit time.

[0120] Among them, the method for generating a video clip based on the current subtext includes: when the user provides an original video (also called a local video), the current subtext is added to the original video as a subtitle to generate a subtitled video, and a subtitled segment is extracted from the subtitled video as a video segment; when the user does not provide the original video, a large language model (LLM) is used to convert the current subtext into a video segment, or a video material matching the current subtext is selected from the material library, subtitles are configured for the video material, and a video segment is generated. Among them, the use of a large language model to convert the current subtext into a video segment is to use AI technology to intelligently generate a video segment based on the current subtext. Among them, the material library can be a third-party material video website, such as Pexels, Xiaohongshu, Douyin, etc.

[0121] The above describes three methods of generating video clips. Users can choose any one of them according to their own needs and preferences to complete the generation of video clips. Through multi-strategy video clip generation, the user's selectivity is increased, allowing users to choose the method that suits them best, providing users with creative flexibility and improving the user experience.

[0122] In S5, if the user provides an original video, video editing technology is used to replace the corresponding part of the original video with each video clip, and after the replacement is completed, the target video is generated.

[0123] If the large language model is used to convert the current sub-text into a video clip, the large language model is continued to be used to merge all the generated video clips into the target video. Specifically, the current sub-text is used as the input of the large language model, and the large language model performs video conversion on the current sub-text and outputs the corresponding video clips. When all the video clips are generated, all the video clips are used as the input of the large language model, and the large language model splices these video clips and performs appropriate transition processing on the spliced ​​parts to generate the final target video.

[0124] If a video material matching the current subtext is selected from the material library, subtitles are configured for the video material, and video clips are generated, then editing software is used to splice all the video clips into a target video.

[0125] Furthermore, the above-mentioned distributed video generation method supports diversified calling interfaces, such as platform direct call interface and API open interface (RESTful API or GraphQL API, etc.). The platform direct call interface allows users to directly call system services through the platform, and the API open interface allows external systems to call system services. Regarding system services, the entire system is split into multiple independent microservices such as data sorting service, scheduling service, and execution service. Each microservice has a single function and clear responsibilities, which is convenient for independent expansion and maintenance. Among them, the data sorting service is used for raw text generation, raw text cleaning, and raw text splitting;

[0126] Reference Figure 5 The schematic diagram of the video generation process is shown. The split text segments are passed to the scheduling service to request the generation of multiple video segment generation tasks (also called subtasks). The scheduling service uses the priority queue to manage the execution order of the subtasks, and uses the full-load queue to assist the priority queue to ensure the sequential execution of the subtasks. The subtasks complete the comprehensive processing of multimedia elements such as video, audio, and subtitles by calling the integrated video generation service (self-built system or third-party system), and notify the scheduling service in real time to update the subtask status after the video segment (audio and subtitle files) is generated. When all subtasks are completed, based on the correct sorting and synthesis of all video segments, the target video generation task (main task) is also completed. The target video is uploaded to OSS or other object storage services through the task completion notification mechanism. Finally, the task status is updated and the caller is notified that the overall task is completed, and relevant task result information is provided. This method of splitting the overall task into a main task and multiple subtasks, with the main task responsible for tracking the overall progress and the subtasks responsible for specific video generation work, realizes flexible task allocation and execution.

[0127] High-availability databases such as MySQL are used for persistent storage of task information, and components such as ETCD are used as registration centers. The registration center is used to monitor information changes of execution nodes. For example, after receiving a subtask, the information of the execution node in the registration center is updated, and the execution node starts to execute the subtask, thereby realizing dynamic management and maintenance of the execution node. The memory cache information in the scheduling service can be updated according to the dynamic information of the execution node to ensure real-time performance.

[0128] In summary, the target video generated by this method has clear pictures, smooth content connection, comprehensive details, and meets the needs of professional-level video production. The generation time of the entire target video is very short, and users can finely control the generated content, for example, specify scenes and character actions, etc., and can also provide rich semantic information, so that the target video contains rich visual, audio and other multimodal information, and effectively integrates this information to improve the quality of video generation. Most of the existing AI text video services are based on single-point model deployment, lacking a complete fault-tolerant mechanism and dynamic expansion capabilities, and are difficult to cope with large-scale user requests and complex application scenarios. The video generation method disclosed in this disclosure adopts a distributed mechanism, which has a good fault-tolerant mechanism and dynamic expansion capabilities, and can adapt to the needs of large-scale user requests and various complex application scenarios.

[0129] Reference Figure 6 The present disclosure provides a distributed video generation system, comprising:

[0130] The splitting module 101 is used to obtain the original text and split the original text into multiple sub-texts;

[0131] A setting module 102, used to set multiple execution nodes and set priorities for the multiple execution nodes;

[0132] A traversal module 103 is used to traverse the multiple subtexts according to their positions in the original text to determine the current subtext;

[0133] An allocation module 104 is used to allocate the current subtext to the execution node with the highest current priority to generate a video segment;

[0134] The synthesis module 105 is used to synthesize all the generated video clips into a target video.

[0135] The various variations and specific examples in the distributed video generation method provided above are also applicable to the distributed video generation system provided in the present disclosure. Through the above detailed description of the distributed video generation method, those skilled in the art can clearly know the implementation method of the distributed video generation system. For the sake of brevity of the specification, it will not be described in detail here.

[0136] The computer device according to the embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.

[0137] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the computer device performs all or part of the steps of the distributed video generation method of each embodiment of the present disclosure.

[0138] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present disclosure.

[0139] like Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure is shown, which is a schematic diagram of the structure of a computer device suitable for implementing the embodiment of the present disclosure. Figure 7 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0140] like Figure 7 As shown, the computer device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0141] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes, hard disks, etc.; and communication devices. The communication device can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 7 A computer device having various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0142] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the distributed video generation method of the embodiment of the present disclosure are executed.

[0143] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0144] According to the computer-readable storage medium of the embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the distributed video generation method of each embodiment of the present disclosure are executed.

[0145] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0146] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0147] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0148] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0149] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0150] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0151] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0152] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0153] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A distributed video generation method, characterized in that: include: Obtaining an original text, and splitting the original text into multiple sub-texts; Setting multiple execution nodes and setting priorities for the multiple execution nodes; According to the sequential positions of the multiple subtexts in the original text, traverse the multiple subtexts to determine the current subtext; Allocating the current subtext to the execution node with the highest current priority to generate a video clip; All generated video clips are synthesized into the target video; The step of splitting the original text into multiple sub-texts includes: Splitting the original text into multiple text segments according to preset symbols; Sort the multiple text fragments according to their positions in the original text to obtain a total number of text fragments; Determine whether the number of characters in the i-th text segment is greater than a preset character threshold; If the number of characters in the i-th text segment is greater than a preset character threshold, or the total number of characters in the i-th text segment and the i+1-th text segment is greater than a preset character threshold, the i-th text segment is taken as a subtext, i=i+1; If the total number of characters of the i-th text segment and the i+1-th text segment is not greater than the preset character threshold, the i-th text segment and the i+1-th text segment are merged to generate a new i-th text segment, and whether the total number of characters of the new i-th text segment and the i+1-th text segment is not greater than the preset character threshold is continued to be determined; the multiple text segments are reordered, and the total number of text segments is updated; When i is greater than the total number of text segments, it indicates that the original text has been split.

2. The distributed video generation method according to claim 1, characterized in that: The assigning priorities to the plurality of execution nodes comprises: Each execution node includes a plurality of sub-nodes, and the sub-nodes are used to generate a video segment according to the assigned current sub-text; Real-time statistics of the number of idle child nodes of each execution node; sorting the plurality of execution nodes according to the number of idle child nodes; The sorting result is used as the priority of the multiple execution nodes.

3. The distributed video generation method according to claim 1, characterized in that: The setting of priorities for the multiple execution nodes includes: Each execution node includes a plurality of sub-nodes, and the sub-nodes are used to generate a video segment according to the assigned current sub-text; Real-time statistics of the number of idle child nodes of each execution node; Based on the number of idle child nodes, obtaining scores of the plurality of execution nodes; sorting the plurality of execution nodes according to the scores; Using the sorting result as the priority of the multiple execution nodes; The calculation formula of the score is as follows: ； Where x is the sequence number of the execution node; j is the sequence number of the historical subtext, and J is the total number of historical subtexts; P x is the score of the xth execution node; w1, w2, and w3 are all preset weight values; F x The number of idle child nodes of the xth execution node; is the execution efficiency of the xth execution node on the jth historical subtext, ;t j Z is the time taken by the xth execution node to generate a video segment based on the jth historical subtext; j is the number of characters in the jth historical subtext; G x is the failure frequency of the xth execution node.

4. The distributed video generation method according to claim 1, characterized in that: Also includes: Each execution node includes a plurality of sub-nodes, and the sub-nodes are used to generate a video segment according to the assigned current sub-text; Real-time statistics of the number of idle child nodes of each execution node; When the execution node is stored in the priority queue and the number of idle child nodes of the execution node is 0, the execution node is moved from the priority queue to the full-load queue without setting the priority of the execution node in the full-load queue; When the execution node is stored in the full-load queue and the number of idle child nodes of the execution node is greater than a preset threshold, or all child nodes of the execution node are idle, the execution node is moved from the full-load queue to the priority queue.

5. The distributed video generation method according to claim 4, characterized in that: Also includes: When the execution node fails, the execution node is removed from the original queue, where the original queue is the priority queue or the full-load queue; Re-prioritize non-faulty execution nodes; Move all subtexts out of the failed execution node; Reassign the removed subtexts according to the new priority level; After the faulty execution node is modified, the modified execution node is stored in the priority queue again.

6. The distributed video generation method according to claim 1, characterized in that: Also includes: When a user provides an original video, the current subtext is added as a subtitle to the original video to generate a subtitle video, and a segment with subtitles is extracted from the subtitle video as a video segment; When the user does not provide the original video, a large language model is used to convert the current subtext into the video clip, or a video material matching the current subtext is selected from a material library, subtitles are configured for the video material, and the video clip is generated.

7. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the distributed video generation method described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the distributed video generation method described in any one of claims 1-6.

9. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images