Work-driven video generation method and device and storage medium

Abstract

The invention relates to a video generation method and device based on work driving and a storage medium, and the method comprises the steps: carrying out the entity analysis of a work, constructing a knowledge graph corresponding to the work, and enabling nodes in the knowledge graph to be used for indicating roles in the work, the relationship among different nodes is used for indicating at least one of the following relationships among different roles: a role relationship and an event relationship; performing plot analysis on the works to obtain a plot library comprising a plurality of atomic plots; a video generation prompt is received, the video generation prompt is used for indicating a plot requirement corresponding to the to-be-generated video, and the plot requirement is related to the works; determining a script corresponding to the video based on the video generation prompt, the knowledge graph and the plurality of atomic plots; a video is generated based on the script. According to the method and the device, the whole-course automatic creation of the second creation works is realized, the video generation efficiency is high, the creation difficulty is low, and the created works have the advantage of high quality due to the high cooperation of all links in the creation process.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a work-driven video generation method, apparatus, and storage medium. Background Technology

[0002] In the field of animated video generation, a high degree of human involvement is typically required. For example, a script must be created manually based on the video generation requirements, and even after receiving the script, manual creation is still necessary to generate the animated video. Currently available automated solutions in animated video generation are not highly feasible, only automating single stages, such as automatically generating a video from a digital resource. However, they cannot automate multiple stages or the pipeline formed by these stages. This leads to numerous problems, including low video generation efficiency, high creative difficulty, and low-quality videos. Summary of the Invention

[0003] This disclosure provides a work-driven video generation method, apparatus, and storage medium to address problems in related technologies. The technical solution of this disclosure is as follows: According to a first aspect of the present disclosure, a work-driven video generation method is provided, the method comprising: By performing entity analysis on the work, a knowledge graph corresponding to the work is constructed. The nodes in the knowledge graph are used to indicate the characters in the work, and the relationships between different nodes are used to indicate at least one of the following relationships between different characters: character relationship and event relationship. The plot of the work was analyzed to obtain a plot library including multiple atomic plots; Receive a video generation prompt, the video generation prompt being used to indicate the plot requirements corresponding to the video to be generated, the plot requirements being related to the work; Based on the video generation prompts, the knowledge graph, and the multiple atomic plots, the script corresponding to the video is determined; The video is generated based on the script.

[0004] In one exemplary implementation, determining the script corresponding to the video based on the video-generated prompts, the knowledge graph, and the multiple atomic plots includes: In the knowledge graph, query the characters, character relationships, and event relationships related to the plot requirements to obtain the first query result; In the plot database, query the atomic plots related to the plot requirement to obtain a second query result; The script is constructed based on the first query result and the second query result.

[0005] In an exemplary implementation, the events corresponding to the event relationships include corresponding first time information, each event corresponds to multiple atomic plots, the plot library is used to record each atomic plot, and the recorded atomic plot includes second time information; the step of constructing the script based on the first query result and the second query result includes: If the first query result includes multiple event relationships, an event chain is obtained based on the temporal relationship indicated by the first time information of the corresponding multiple events; Based on the consistency between each atomic plot in the second query result and the event in the event chain, an atomic plot sequence is determined. The consistency includes time consistency determined based on the first time information and the second time information, and semantic consistency determined based on the semantics of the atomic plot and the semantics of the event. The script is constructed based on the atomic plot sequence, other information in the first query result, and the video generation prompts.

[0006] In one exemplary embodiment, the process of performing plot analysis on the work to obtain a plot library including multiple atomic plots includes: Obtain atomic story templates corresponding to multiple story types. The atomic story templates are used to define the metadata corresponding to the story type. The metadata refers to the control attribute data required to present the atomic story of the corresponding story type. Based on the atomic plot template, each event in the work is split into plot segments, and the atomic plot template is filled based on the splitting results to obtain multiple atomic plots corresponding to the event; The multiple atomic plots are stored in the plot library.

[0007] In one exemplary implementation, storing the plurality of atomic storylines in the storyline library includes: For any given atomic plot, extract the plot summary corresponding to that atomic plot; Feature extraction is performed on the plot summary to obtain the plot representation information corresponding to the atomic plot; Based on the plot summary, the atomic plot, and the plot representation information, construct the plot record corresponding to the atomic plot; The plot records are saved in the plot library.

[0008] In one exemplary implementation, the step of querying the plot database for atomic plots related to the plot requirement to obtain a second query result includes: In the video generation prompt, extract the first prompt information used to indicate the requirements of the atomic plot; Extract the semantic representation information corresponding to the first prompt information; Based on the semantic representation information, a first number of target plot representation information are queried in the plot library, the first number being determined based on the video duration corresponding to the video. The second query result is obtained based on the atomic plots corresponding to each of the plot representation information.

[0009] In one exemplary implementation, determining the atomic plot sequence based on the correspondence between each atomic plot in the second query result and an event in the event chain includes: Based on the semantic consistency between each atomic plot and each event in the event chain, a second number of atomic plots are determined; The atomic plot sequence is determined based on the second time information of each of the second number of atomic plots.

[0010] In one exemplary implementation, generating the video based on the script includes: For the entities in the script, extract the entity descriptions corresponding to the entities from the knowledge graph; Based on the entity description, determine the control attributes of the entity corresponding to multiple modalities; Based on the control attributes corresponding to the multiple modalities, determine the digital resources of the entity in the multiple modalities; The video is generated based on the entity's digital resources in multiple modalities.

[0011] In one exemplary implementation, generating the video based on the entity's digital resources in multiple modalities includes: Based on a large-scale generative language model, the script is semantically parsed and processed into scene sequences to obtain a scene sequence. The video is generated based on the digital resources of each entity in multiple modalities and the storyboard sequence.

[0012] According to a second aspect of the present disclosure, a work-driven video generation apparatus is provided, the apparatus comprising: The entity analysis module is configured to perform entity analysis on the work and construct a knowledge graph corresponding to the work. The nodes in the knowledge graph are used to indicate the characters in the work, and the relationships between different nodes are used to indicate at least one of the following relationships between different characters: character relationship and event relationship. The plot library construction module is configured to perform plot analysis on the work to obtain a plot library including multiple atomic plots; The prompt receiving module is configured to execute a video generation prompt, wherein the video generation prompt is used to indicate the plot requirements corresponding to the video to be generated, and the plot requirements are related to the work. The script generation module is configured to determine the script corresponding to the video based on the video generation prompts, the knowledge graph, and the multiple atomic plots; The video generation module is configured to generate the video based on the script.

[0013] In one exemplary implementation, the script generation module is configured to execute: In the knowledge graph, query the characters, character relationships, and event relationships related to the plot requirements to obtain the first query result; In the plot database, query the atomic plots related to the plot requirement to obtain a second query result; The script is constructed based on the first query result and the second query result.

[0014] In one exemplary implementation, the events corresponding to the event relationships include corresponding first time information, each event corresponds to multiple atomic plots, the plot library is used to record each atomic plot, and the recorded atomic plot includes second time information; the script generation module is configured to execute: If the first query result includes multiple event relationships, an event chain is obtained based on the temporal relationship indicated by the first time information of the corresponding multiple events; Based on the consistency between each atomic plot in the second query result and the event in the event chain, an atomic plot sequence is determined. The consistency includes time consistency determined based on the first time information and the second time information, and semantic consistency determined based on the semantics of the atomic plot and the semantics of the event. The script is constructed based on the atomic plot sequence, other information in the first query result, and the video generation prompts.

[0015] In one exemplary implementation, the plot library building module is configured to perform: Obtain atomic story templates corresponding to multiple story types. The atomic story templates are used to define the metadata corresponding to the story type. The metadata refers to the control attribute data required to present the atomic story of the corresponding story type. Based on the atomic plot template, each event in the work is split into plot segments, and the atomic plot template is filled based on the splitting results to obtain multiple atomic plots corresponding to the event; The multiple atomic plots are stored in the plot library.

[0016] In one exemplary implementation, the plot library building module is configured to perform: For any given atomic plot, extract the plot summary corresponding to that atomic plot; Feature extraction is performed on the plot summary to obtain the plot representation information corresponding to the atomic plot; Based on the plot summary, the atomic plot, and the plot representation information, construct the plot record corresponding to the atomic plot; The plot records are saved in the plot library.

[0017] In one exemplary implementation, the script generation module is configured to execute: In the video generation prompt, extract the first prompt information used to indicate the requirements of the atomic plot; Extract the semantic representation information corresponding to the first prompt information; Based on the semantic representation information, a first number of target plot representation information are queried in the plot library, the first number being determined based on the video duration corresponding to the video. The second query result is obtained based on the atomic plots corresponding to each of the plot representation information.

[0018] In one exemplary implementation, the script generation module is configured to execute: Based on the semantic consistency between each atomic plot and each event in the event chain, a second number of atomic plots are determined; The atomic plot sequence is determined based on the second time information of each of the second number of atomic plots.

[0019] In one exemplary implementation, the video generation module is configured to perform: For the entities in the script, extract the entity descriptions corresponding to the entities from the knowledge graph; Based on the entity description, determine the control attributes of the entity corresponding to multiple modalities; Based on the control attributes corresponding to the multiple modalities, determine the digital resources of the entity in the multiple modalities; The video is generated based on the entity's digital resources in multiple modalities.

[0020] In one exemplary implementation, the video generation module is configured to perform: Based on a large-scale generative language model, the script is semantically parsed and processed into scene sequences to obtain a scene sequence. The video is generated based on the digital resources of each entity in multiple modalities and the storyboard sequence.

[0021] According to a third aspect of the present disclosure, an electronic device is provided, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the work-driven video generation method as described in any of the above embodiments.

[0022] According to a fourth aspect of the present disclosure, a computer storage medium is provided, wherein when instructions in the computer storage medium are executed by a processor of an electronic device, the electronic device performs the work-driven video generation method described in any of the above embodiments.

[0023] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including a computer program that, when executed by a processor, implements the work-driven video generation method described in any of the above embodiments.

[0024] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: The work-driven video generation method provided in this disclosure can automatically understand the work, construct a knowledge graph and a plot library for managing atomic plots. Based on the plot requirements recorded in the received video generation prompts, it automatically constructs a script that meets the plot requirements using information from the knowledge graph and plot library, and automatically generates a video based on the script. This achieves multi-stage automation of the work understanding stage, script generation stage, and script-based video generation stage. The work understanding stage, script generation stage, and script-based video generation stage form a complete pipeline from the original work to the final video (secondary creative work). Therefore, this disclosure essentially automates the entire pipeline of secondary creation, achieving fully automated creation of secondary creative works. Videos generated through this fully automated creation process require no manual intervention, have high generation efficiency, low creation difficulty, and the high degree of coordination among the various stages of the creation process results in high-quality works.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0027] Figure 1 This is a flowchart illustrating a work-driven video generation method according to an exemplary embodiment; Figure 2 This is a schematic diagram of a plot library construction method according to an exemplary embodiment. Figure 1 ; Figure 3 This is a schematic diagram of a plot library construction method according to an exemplary embodiment. Figure 2 ; Figure 4 This is a schematic diagram illustrating a script determination method according to an exemplary embodiment; Figure 5 This is a schematic diagram illustrating a script construction method according to an exemplary embodiment; Figure 6 This is a plot library query method illustrated according to an exemplary embodiment; Figure 7 This is a block diagram of a work-driven video generation apparatus according to an exemplary embodiment; Figure 8 This is a structural block diagram of a computer device according to an exemplary embodiment. Figure 1 ; Figure 9 This is a structural block diagram of a computer device according to an exemplary embodiment. Figure 2 . Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0029] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0030] To facilitate understanding of this disclosure, a brief introduction to the background technology of this disclosure is provided first: AIGC stands for Artificial Intelligence Generated Content. This concept encompasses the automatic generation of various forms of content, such as text, images, audio, and video, using artificial intelligence technology. In practical applications, AIGC can significantly improve the efficiency of content creation, reduce labor costs, and provide creators with more inspiration and possibilities. However, current AIGC animation solutions mostly focus on single-point generation, such as generating videos based on text or images. But single-point generation solutions have the following problems when applied to the secondary creation (derivative creation) of original works: 1. The lack of a deep, structured understanding of the worldview of the original work leads to numerous problems in the generated video, such as confusion in character relationships and spatiotemporal logic.

[0031] 2. The style of a character drifts in different shots, meaning that the same character looks very different in different shots.

[0032] 3. The generated video's plot has a low match with the user's plot requirements, making it difficult to obtain high-quality derivative videos based on the user's plot requests. For example, it is difficult for a user to input "want to see Zhuge Liang capture Meng Huo seven times" and immediately get a derivative video expressing the corresponding story. This is because the relevant technology lacks the ability to deeply understand the work, sort out the plot, extract complex plots, and create derivative works. The "seven captures of Meng Huo" is a plot that is sorted out and summarized from the overall work, not the initial plot that is directly and explicitly recorded in the work. This means that the relevant technology cannot automatically obtain "seven captures of Meng Huo," resulting in a low match between the generated video's plot and the user's plot requirements.

[0033] To address these technical issues, this disclosure provides a work-driven video generation method. This method achieves a fully automated execution scheme from the original work to the derivative video by deeply understanding the original work, thereby solving the aforementioned problems.

[0034] Figure 1This is a flowchart illustrating a work-driven video generation method according to an exemplary embodiment. The work-driven video generation method can be applied to electronic devices, which can be implemented independently by a server or a terminal, or collaboratively by a terminal and a server. The terminal can be, but is not limited to, physical devices such as smartphones, tablets, laptops, desktop computers, smart speakers, smart wearable devices, digital assistants, augmented reality devices, and virtual reality devices, and can also include software such as applications running on the physical device. The server can be, but is not limited to, a standalone server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms, etc. (Refer to...) Figure 1 As shown, the method includes the following steps.

[0035] In S110, by performing entity analysis on the work, a knowledge graph corresponding to the work is constructed. The nodes in the knowledge graph are used to indicate the characters in the work, and the relationships between different nodes are used to indicate at least one of the following relationships between different characters: character relationship and event relationship.

[0036] Entity analysis refers to the process of extracting entities with independent meaning from a work and classifying and clarifying the relationships between these entities. In this process, core elements (entities) of the work, such as characters, events, plots, and scenes, can be identified and their relationships analyzed. For example, in a novel, in-depth textual analysis can extract entities such as main characters, important scenes, and key plot points. These entities not only form the basis of a knowledge graph but also reveal the internal logical structure of the work through their interrelationships. For instance, kinship or antagonistic relationships between characters are marked as relationships between nodes, while the occurrence of a certain event may connect multiple character nodes, thus forming a multi-dimensional semantic network. This analytical method helps to more comprehensively understand the content of the work and its internal connections, and reflects the results of this understanding on a knowledge graph.

[0037] This disclosure does not limit the execution scheme of S110. For example, entity analysis and knowledge graph construction can be achieved using large-scale generative language models. For instance, the text of a work can be segmented and tagged with parts of speech to identify entities, such as key elements like characters, locations, and events. These entities will become nodes in the knowledge graph. Then, natural language understanding technology is used to analyze the relationships between entities, extracting role relationships and event relationships, and connecting these relationships to the graph as edges. For example, when analyzing *Romance of the Three Kingdoms*, the brotherhood between "Liu Bei" and "Guan Yu," or the association between the Battle of Red Cliffs and "Cao Cao's" defeat, can be identified.

[0038] In some exemplary implementations, a context-aware mechanism can be introduced to improve the accuracy of the knowledge graph. By combining contextual information, the model can more accurately determine the type of relationship between entities. For example, in different plot segments, the relationship between "Zhuge Liang" and "Sima Yi" may exhibit different characteristics such as hostility, intellectual struggle, or strategic contest.

[0039] In one implementation, taking the work "Romance of the Three Kingdoms" as an example, in S110, "Romance of the Three Kingdoms" can be input into a large-scale generative language model, triggering the large-scale generative language model to construct a knowledge graph of the entire text. This knowledge graph can represent the main characters, scenes, events, and other entities in "Romance of the Three Kingdoms," as well as the relationships between them. The process of executing S110 can be represented as follows: Input: Text of Chapter 120 of Romance of the Three Kingdoms Large generative language models perform the following steps: (1) Chapter splitting and converting classical Chinese into standard Chinese: When splitting text into chapters, the boundaries of each chapter are determined based on the natural paragraphs and plot turning points of the text. This method ensures that the generated content conforms to the narrative rhythm of the original work while facilitating subsequent processing. For example, in *Romance of the Three Kingdoms*, chapters can be divided based on major battles, turning points in characters' fates, or important dialogues. Simultaneously, to improve the model's ability to understand classical Chinese, parallel corpus fine-tuning techniques can be introduced. By comparing translation pairs of classical Chinese with modern vernacular Chinese, the model is trained to perform language conversion tasks more accurately.

[0040] Next, in the process of translating Classical Chinese into Standard Mandarin, through the study of a large number of classic literary works and their modern translations, the model can master the sentence structure and vocabulary usage of Classical Chinese and transform them into fluent and easy-to-understand Standard Mandarin expressions. This process not only needs to preserve the core semantics of the original text, but also needs to take into account the tone and emotion conveyed, so as to ensure that the final generated content can faithfully reflect the style and connotation of the original work.

[0041] Of course, if the original work is written in Mandarin, there is no need to convert it from classical Chinese to Mandarin. If the original work is written in a different language, such as a foreign language, then it is necessary to convert it from a foreign language to Mandarin. This process will not be elaborated on, as it is based on the same invention concept as converting classical Chinese to Mandarin.

[0042] (2) Role-relationship extraction Character types: Zhuge Liang, Cao Cao, Zhao Yun, Guan Yu, etc.; Each character type can possess attributes such as gender, appearance, weapon, clothing, personality traits, and faction affiliation. These attributes will provide the foundation for subsequent plot generation, ensuring that character behavior aligns with the established character design.

[0043] If certain events occur between different characters, the events and the relationships between the characters within those events are recorded, such as the cause, process, and result of the events, as well as the role and attitude of each character in the events. For example, these relationships can include various types such as cooperation, conflict, alliance, and betrayal. These relationships not only reflect the interaction patterns between characters but also provide driving forces for the plot development. For example, the cooperation between Zhuge Liang and Liu Bei promoted the establishment of the Shu Han state, while the conflict between Cao Cao and Yuan Shao became the core thread of the Battle of Guandu.

[0044] Furthermore, it extracts the relationships between different events. For example, different events may have causal or temporal relationships; for instance, the occurrence of event A may lead to the occurrence of event B. This causal relationship can support the coherence of the plot. At the same time, some events may have a clear chronological order, such as event C must occur before event D. This temporal relationship helps to construct a clear story timeline. By analyzing these relationships, the event connection logic in the knowledge graph can be further optimized to ensure that the generated plot not only conforms to the historical background or story setting but also has a natural and smooth narrative rhythm.

[0045] It can also extract the relationships between different roles, such as ruler and subject, allies, enemies, and affiliations.

[0046] (3) Knowledge graph construction For example, the node corresponding to the entity is shown below: (Zhuge Liang: Person {Courtesy name: "Kongming", Faction: "Shu", Skills: "Fire Attack, Eight Trigrams Formation", Appearance: "Feather Fan and Silk Turban, Riding a Four-Wheeled Chariot"}) For each event, its time attribute also needs to be recorded, such as the 13th year of Jian'an, the 3rd year of Jianxing, etc., to support chronological reasoning. In addition, it can detect whether there are conflicts in the knowledge graph, that is, to perform conflict and consistency checks. For example, if a node still appears after "Guan Yu died in 219" is automatically detected, manual intervention is required to correct it.

[0047] S120. Perform plot analysis on the work to obtain a plot library including multiple atomic plots.

[0048] An event may include multiple stages such as cause, process, and result. Each stage can be represented by multiple atomic plots. An atomic plot is understood to be the smallest unit of information that can be visually represented. This disclosure does not limit the method of parsing atomic plots or the method of constructing the plot library. For example, large-scale generative language models can be used for plot parsing and atomic plot construction.

[0049] In one exemplary implementation, please refer to Figure 2 This illustrates a plot library construction method according to an exemplary embodiment of the present disclosure. Figure 1 The process of analyzing the plot of the work yields a plot library containing multiple atomic plots, including: S210. Obtain atomic plot templates corresponding to multiple plot types, wherein the atomic plot templates are used to define the metadata corresponding to the plot type, and the metadata refers to the control attribute data required to present the atomic plot of the corresponding plot type; This disclosure does not limit the plot type; for example, the plot type and the corresponding atomic plot template can be represented as: T1 Duel Type: A and B engage in battle at location X → Victory / Defeat / Recruitment T2 Strategy Type: A uses strategy Y to change the course of the battle. T3 Emotional Type: Dialogue between A and B → Relationship Change ... Metadata can be used to indicate the characteristics of information related to the modalities that present the atomic plot, thereby controlling the generation of corresponding modal data. For example, it can indicate whether there are ethnic minority customs, emotional tone, emotional characteristics, geographical attributes, character emotional expressions, plot geographical features, plot character special effects, key lines, etc.

[0050] Taking the Battle of Red Cliffs as an example, a typical atomic scenario in the Battle of Red Cliffs could be "T2 Strategy Type: A uses Y's strategy → changes the course of the battle." Specifically, Zhuge Liang's borrowing arrows with straw boats is an example. In this atomic scenario, the metadata includes the following: the emotional tone is one of tense and intellectually stimulating confrontation; the geographical attribute is the Yangtze River region; the characters' emotions are Zhuge Liang's composure and Zhou Yu's anxiety; and the key line is Zhuge Liang's words to Zhou Yu, "Everything is ready, only the east wind is lacking." Furthermore, the scenario involves specific visual effects, such as the dense array of ships on the river and a rain of arrows that blots out the sky, further enhancing the dramatic tension of the strategic execution. This metadata collectively constitutes a comprehensive description of the atomic scenario, providing foundational information for the subsequent generation of multimodal digital assets.

[0051] S220. Based on the atomic plot template, each event in the work is split into plot segments, and the atomic plot template is filled based on the splitting results to obtain multiple atomic plots corresponding to the event.

[0052] The various information involved in the atomic storyline is populated into the atomic storyline template. This template defines metadata and any other necessary information describing the atomic storyline. For example, the template may include key fields such as storyline type identifier, participating characters, trigger conditions, action descriptions, and outcome impacts. In this way, each event can be systematically broken down into multiple atomic storyline units, facilitating subsequent structured processing and multimodal generation.

[0053] S230. Save the multiple atomic plots in the plot library.

[0054] This disclosure allows for the structured representation of each atomic plot element, and the representation results can be stored to obtain a plot library corresponding to the work. For example, taking "Romance of the Three Kingdoms" as an example, all atomic plot elements in the entire text need to be extracted and stored in the library, resulting in a plot library containing hundreds of thousands of atomic plot elements.

[0055] By extracting core storylines from a work and constructing a storyline library encompassing these individual storylines, suitable storylines can be readily retrieved based on the requirements of the adaptation. These storylines can then be re-created and reorganized to achieve innovative adaptations of the original work. The flexible use of this storyline library allows for the rapid construction of new storylines while preserving the core style and tone of the original work. Furthermore, the storyline library provides rich material support for multimodal generation, ensuring that subsequent character dialogue generation, scene descriptions, and emotional rendering are more closely aligned with the original context, laying a solid foundation for a high-quality adaptation.

[0056] In one exemplary implementation, please refer to Figure 3This illustrates a plot library construction method in an exemplary embodiment of this disclosure. Figure 2 The step of storing the multiple atomic storylines in the storyline library includes: S310. For any atomic plot, extract the plot summary corresponding to the atomic plot; perform feature extraction on the plot summary to obtain the plot representation information corresponding to the atomic plot; construct the plot record corresponding to the atomic plot based on the plot summary, the atomic plot and the plot representation information; The purpose of extracting plot summaries and constructing plot representation information is to ensure that each atomic plot can be accurately described and efficiently retrieved. Extracting plot summaries concisely summarizes the core content of each atomic plot, facilitating quick review and retrieval. Constructing plot representation information aims to extract more semantically valuable features through in-depth analysis of the summaries, providing query support for the plot library. This disclosure does not limit the feature extraction method; for example, it can be implemented using open-source semantic extraction models or self-trained semantic extraction models.

[0057] This disclosure does not limit the plot summary, which may include the plot style, plot features, plot type, plot focus, key plot characters, the stage of the plot in the overall event (cause, process, result), and the role of the plot in the overall event (plot advancement, climax), etc.

[0058] S320. Save the plot record in the plot library.

[0059] By generating plot records that include three elements—plot summary, plot representation information, and atomic plots—the plot library can support efficient retrieval of atomic plots by storing structured plot records. For example, this allows the plot library to support semantic search and tag search. These tags typically define information such as plot style, plot features, and plot type. This information falls into the plot summary, thus enabling the plot library to support tag-based retrieval.

[0060] S130. Receive a video generation prompt, the video generation prompt being used to indicate the plot requirements corresponding to the video to be generated, the plot requirements being related to the work.

[0061] The video generation prompts come from user input and are a natural expression of the user's secondary creative ideas. The video generation prompts can be text information or audio information. This disclosure can convert the video generation prompts in audio state into text state for subsequent processing.

[0062] For example, if a user enters "I want to watch a condensed version of Zhuge Liang's seven captures of Meng Huo, 6 episodes in total, 2 minutes per episode", then "a condensed version of Zhuge Liang's seven captures of Meng Huo, 6 episodes in total, 2 minutes per episode" will be the video generation prompt.

[0063] S140. Based on the video generation prompts, the knowledge graph, and the multiple atomic plots, determine the script corresponding to the video.

[0064] The system analyzes key information from video prompts, such as core plot events, character relationships, and the expected narrative pacing. Utilizing information from knowledge graphs and a plot database, it automatically re-creates the script based on this key information.

[0065] In one exemplary implementation, please refer to Figure 4 The diagram illustrates a script determination method in an exemplary embodiment of this disclosure. The step of determining the script corresponding to the video based on the video generation prompts, the knowledge graph, and the multiple atomic plots includes: S410. In the knowledge graph, query the characters, character relationships, and event relationships related to the plot requirements to obtain the first query result; The first query result includes various information related to the plot requirements from the knowledge graph, including characters, character relationships, and event relationships. Specifically, for each character, it can include their own character profile, such as gender, age, physical characteristics, special effects features, personality traits, and their role in the plot requirements. Character relationships cover the degree of closeness between characters, their alliance or hostility, and how these relationships change as the plot develops. Event relationships detail the causal connections, temporal order, and logical dependencies between various plot-related events. For example, in the plot of Zhuge Liang capturing Meng Huo seven times, it can include event relationships related to Meng Huo's multiple resistances and failures. Furthermore, the first query result can also record implicit information, such as the influence of environmental factors on events, or the role of certain minor characters at crucial moments, thus providing richer and more detailed material support for subsequent script generation.

[0066] S420. In the plot database, query the atomic plots related to the plot requirement to obtain a second query result; The second query result includes individual atomic plots that can be used to build a plot that fits the script. These atomic plots can be used as material for creating the script, thereby narrowing down the range of materials and eliminating the need to search for materials in the original work.

[0067] S430. Construct the script based on the first query result and the second query result.

[0068] This disclosure automatically constructs a script by analyzing character profiles, character relationships, and event relationships from the first query result, combined with atomic plots provided by the second query result. Based on the first query result, it provides a macro-level understanding of the event's development direction. Simultaneously, leveraging the modular nature of atomic plots, it quickly assembles storylines that meet the plot requirements, thus avoiding inconsistencies such as plot contradictions that may arise during the creation process, significantly improving the efficiency and quality of script generation. On one hand, the introduction of knowledge graphs makes script creation more data-driven, reducing subjective biases from human intervention; on the other hand, the flexible combination of atomic plots reduces reliance on original materials, expanding the creative possibilities. Furthermore, through detailed character and event portrayals, the generated script is more realistic and immersive, contributing to enhanced artistic expression and audience acceptance of the final work.

[0069] In one exemplary implementation, the events corresponding to the event relationships include corresponding first time information, each event corresponds to multiple atomic plots, the plot library is used to record each atomic plot, and the recorded atomic plots include second time information.

[0070] First-time information refers to the time when an event occurs, while second-time information refers to the time when the corresponding atomic events occur, or the chronological relationship between different atomic events of the same event. For example, in *Romance of the Three Kingdoms*, the Battle of Red Cliffs is a crucial event, and its first-time information can be defined as the winter of the thirteenth year of Jian'an (208 AD). This event contains multiple atomic events, such as "Zhuge Liang borrowing arrows with straw boats," "Huang Gai's feigned surrender," and "Zhou Yu's fire attack." The order and time intervals of these atomic events constitute the second-time information. This kind of time division allows for a clearer understanding of the logical chain of events, while ensuring the reasonable distribution of each atomic event along the timeline.

[0071] Please refer to Figure 5 The diagram illustrates a script construction method according to an exemplary embodiment of this disclosure. The step of constructing the script based on the first query result and the second query result includes: S510. If the first query result includes multiple event relationships, an event chain is obtained based on the temporal relationship indicated by the first time information of the corresponding multiple events; Taking the "Seven Captures of Meng Huo" scenario required by the script as an example, this event chain can be arranged chronologically to trace the specific process of each capture and release. For instance, the first capture might occur when Meng Huo led his troops to invade the Shu army camp, and its first moment of information corresponds to the time of the initial confrontation between the two sides; while the second might be when Zhuge Liang devised a plan to lure the enemy deep into his territory and then captured Meng Huo again... This event chain includes a total of seven captures of Meng Huo, and these events are arranged in the order of their occurrence.

[0072] S520. Based on the consistency between each atomic plot in the second query result and the event in the event chain, determine the atomic plot sequence. The consistency includes the time consistency determined based on the first time information and the second time information, and the semantic consistency determined based on the semantics of the atomic plot and the semantics of the event.

[0073] Semantic consistency refers to the semantic matching relationship between each atomic plot and the events in the event chain. Based on semantic consistency, it can be determined which event each atomic plot belongs to. Temporal consistency includes two aspects: one is to determine which event each atomic plot belongs to based on the occurrence time of each atomic plot and the occurrence time of each event; the other is to determine the relative temporal relationship of the occurrence times of atomic plots belonging to the same event, thereby sorting these atomic plots. This ensures that the sorted atomic plot sequence not only reflects the sequential occurrence of each event, but also effectively avoids plot confusion or logical inconsistencies that may occur during secondary creation in related technologies. For example, in the construction of the "Seven Captures of Meng Huo" scenario, a certain atomic plot describes the plot of Zhuge Liang's plan to lure the enemy deep into the territory. Its semantics are highly consistent with the event of the second capture of Meng Huo. At the same time, the occurrence time of this atomic plot is also consistent with the time node of the second capture in the event chain. Therefore, it can be determined that this atomic plot belongs to the second capture event. Furthermore, for multiple atomic plots belonging to the same event, their order can be further refined based on their temporal information, ensuring that the final generated sequence of atomic plots conforms to both temporal logic and semantic coherence. This two-dimensional matching mechanism provides a reliable foundation for the automatic generation of subsequent scripts.

[0074] S530. Construct the script based on the atomic plot sequence, other information in the first query result, and the video generation prompt.

[0075] The atomic plot sequence provides the basic framework for the story's development, ensuring the script maintains coherence in terms of timeline and logic. Other information in the first query result may include supplementary content such as character relationships, scene descriptions, and emotional changes, enriching the script's details and making it more vivid and three-dimensional. Video generation prompts inject specific requirements into the script. For example, a prompt like "Want to watch a condensed version of Zhuge Liang's seven captures of Meng Huo, 6 episodes, 2 minutes each" includes specific requirements such as duration, number of episodes, and style. Integrating this information allows for automatic script creation; for instance, this information can be input into a large generative language model, instructing it to construct the script.

[0076] For example, the scriptwriting process, which generates prompts based on user-input video, includes: The user entered: "I want to see a condensed version of Zhuge Liang's seven captures of Meng Huo, 6 episodes in total, 2 minutes per episode." Automated execution: The video is analyzed to generate prompts, resulting in a structure that carries the user's creative intent: {Main Character: Zhuge Liang, Event: Seven Captures of Meng Huo, Style: Action, Number of Episodes: 6, Duration: 2 min}. Based on knowledge graph-based event chain retrieval: First capture → Second capture → ... → Seventh capture Retrieve multiple atomic plots based on the plot database; Atomic plot sequences are obtained based on temporal and semantic consistency. Perform consistency checks on the atomic plot sequence: check Meng Huo's combat power, the logic of the rattan armor soldiers' appearance, and automatically fill in transition paragraphs, etc.

[0077] In one exemplary implementation, please refer to Figure 6 This illustrates an exemplary embodiment of a plot library query method. The method involves querying the plot library for atomic plots related to the plot requirement to obtain a second query result, including: S610. From the video generation prompt, extract the first prompt information used to indicate the atomic plot requirements; For example, the first prompt can indicate the requirements for plot style and / or plot semantics. Taking the video generation prompt as "A condensed version of Zhuge Liang's seven captures of Meng Huo, 6 episodes, 2 minutes each," the plot style requirements and plot semantics requirements are "hot-blooded" and "captures Meng Huo," respectively.

[0078] S620. Extract the semantic representation information corresponding to the first prompt information; Since the plot database records the semantic representation information of the plot summaries of each atomic plot, after extracting the semantic representation information corresponding to the first prompt information, multiple atomic plots can be found based on the similarity of the semantic representation information.

[0079] S630. Based on the semantic representation information, query a first number of target plot representation information in the plot library, wherein the first number is determined based on the video duration corresponding to the video; Video length can be obtained through video-generated prompts or by using pre-set methods. The required amount of target plot representation information (the first quantity) is determined based on the video length. For example, longer videos may require more plot support to ensure richness and coherence, while shorter videos need to have a reduced amount of target plot representation information to avoid plot piling up and dragging out the pace. In this way, atomic plots with the highest semantic representation information matching degree can be intelligently filtered out and sorted according to priority. This improves the efficiency and accuracy of plot retrieval. On the one hand, queries based on semantic representation information can effectively filter irrelevant plots; on the other hand, dynamically adjusting the number of target plots in conjunction with video length makes the generated storylines more in line with actual needs, avoiding content imbalance caused by too much or too little plot.

[0080] S640. Based on the atomic plots corresponding to each of the plot representation information, the second query result is obtained.

[0081] In some implementations, determining the atomic plot sequence based on the consistency between each atomic plot in the second query result and the events in the event chain includes: determining a second number of atomic plots based on the semantic consistency between each atomic plot and each event in the event chain; and determining the atomic plot sequence based on the second time information of each of the second number of atomic plots.

[0082] The first number is greater than the second number because, based on the semantic representation information, querying the first number of atomic plots in the plot library constitutes the initial screening of atomic plots. Then, further, based on the semantic consistency of time within the specific event chain, a second number of atomic plots are selected. These second number of atomic plots will be presented in the final generated video, and this second number is also determined based on the video duration. This dual mechanism of initial and secondary screening ensures that the selected atomic plots not only highly match the event chain semantically but also maintain a reasonable distribution on the timeline, avoiding overly dense or sparse plots. This refined screening process also reduces the introduction of redundant plots, thereby improving the overall watchability of the generated video. Simultaneously, sorting the atomic plots based on time information further enhances the temporal coherence of the storyline, allowing viewers to understand the plot development more naturally.

[0083] S150. Generate the video based on the script.

[0084] The work-driven video generation method provided in this disclosure can automatically understand the work, construct a knowledge graph and a plot library for managing atomic plots. Based on the plot requirements recorded in the received video generation prompts, it automatically constructs a script that meets the plot requirements using information from the knowledge graph and plot library, and automatically generates a video based on the script. This achieves multi-stage automation of the work understanding stage, script generation stage, and script-based video generation stage. The work understanding stage, script generation stage, and script-based video generation stage form a complete pipeline from the original work to the final video (secondary creative work). Therefore, this disclosure essentially automates the entire pipeline of secondary creation, achieving fully automated creation of secondary creative works. Videos generated through this fully automated creation process require no manual intervention, have high generation efficiency, low creation difficulty, and the high degree of coordination among the various stages of the creation process results in high-quality works.

[0085] In one exemplary implementation, generating the video based on the script includes: extracting entity descriptions corresponding to entities in the script from the knowledge graph; determining control attributes corresponding to the entity in multiple modalities based on the entity descriptions; determining digital resources of the entity in multiple modalities based on the control attributes corresponding to the multiple modalities; and generating the video based on the digital resources of the entity in multiple modalities.

[0086] For example, a character's physical characteristics, personality traits, and behavioral habits can all be extracted as part of the entity description. Next, based on these descriptions, the entity's performance in different modalities can be analyzed. Modality refers to the mode of expression in visual, auditory, and other sensory dimensions. For example, in the visual modality, a character might require a specific image design; in the auditory modality, it might involve their timbre, tone of voice, signature sound effects, or catchphrases, which need to be implemented using the corresponding modal control attributes. After determining the entity's control attributes in each modality, the next step is to match or generate corresponding digital resources. These resources can include pre-set character models, motion capture data, voice-over files, and background music. In this way, a complete multimodal asset portfolio can be generated for each entity. These digital resources are then arranged and integrated according to the needs of the storyline to form coherent video content. This process not only ensures the consistency of each entity's performance across different shots but also enhances the realism and immersion of the final video work.

[0087] In one exemplary implementation, generating the video based on the digital resources of the entities in multiple modalities includes: performing semantic parsing and storyboarding on the script based on a large-scale generative language model to obtain a storyboard sequence; and generating the video based on the digital resources of each entity in multiple modalities and the storyboard sequence.

[0088] By analyzing key plot points and character actions in the script, the perspective, duration, and transitions of each shot can be determined. For example, when the narrative text describes a tense chase scene, a sequence of shots containing rapidly switching close-ups and extreme close-ups might be generated to enhance the audience's tension. Furthermore, the language style of the shots can be adjusted according to the emotional tone of the plot; for instance, soft lighting effects and slow camera movements can be used in warm scenes. This approach not only improves the efficiency of shot generation but also ensures the artistry and narrative coherence of the video creation.

[0089] In one exemplary implementation, a large-scale generative language model can be used to literaryally expand and refine the script, generating target text with more details. Then, the plot can be decomposed based on the target text and storyboarded according to the decomposition results, or the plot can be decomposed and storyboarded through interaction with the large-scale generative language model. This disclosure does not limit the specific decomposition method, decomposition results, or the process of obtaining a storyboard sequence based on the decomposition results. For example, key plot points and character interactions in the target text can be analyzed to extract core elements suitable for visual expression. These core elements are then organized into a logically coherent storyboard sequence to provide a clear guiding framework for subsequent animated video production, ensuring that the final generated video conforms to both the semantics of the script and the needs of visual narrative. This disclosure does not limit the core elements; for example, these core elements may include character actions, scene transitions, emotional changes, and the use of key props. When generating the storyboard sequence, the details of each storyboard are dynamically adjusted according to the semantic features of the target text, such as the choice of camera angle, control of the scene rhythm, and setting of visual focus. This design not only improves the efficiency of storyboard generation, but also enhances the expressiveness and narrative depth of the final animated video. This disclosure allows storyboard artists to automatically realize manual storyboard design without the need for manual storyboarding. Through automated and professional storyboard decomposition, higher quality storyboard results are obtained.

[0090] Furthermore, based on the storyboard information corresponding to each storyboard in the storyboard sequence, video can be generated by using the entity's digital resources in multiple modalities. Specifically, a track arrangement instruction can be created based on the storyboard information and the digital resources; the track arrangement instruction is sent to the Unreal Engine's track arrangement service, which performs the following operations: Create a track.

[0091] The process of creating a track can be understood as building a basic framework that carries all animation and interactive elements.

[0092] Mount the digital resource.

[0093] Mounting digital resources is the process of binding pre-made digital resources to a corresponding track.

[0094] A timeline is created based on the storyboard information. Based on the timeline, the digital resources are arranged on the track, and the track arrangement result is output.

[0095] The resulting track arrangement is rendered to obtain a video.

[0096] For example, character descriptions can be extracted from a knowledge graph, converted into prompts for image generation, and then used to generate character images using image generation algorithms. Alternatively, target speech information, such as timbre, emotion, and dialect, can be determined based on character descriptions in the knowledge graph. Based on this target language information and lines from the script, audio generation technology can be used to generate the character's speech. All of these fall under the category of multimodal digital resources for characters.

[0097] For example, the multimodal digital resources for the characters Zhuge Liang and Meng Huo can be characterized as follows:

[0098] Next, a large-scale generative language model can be used to segment the script into scenes and expand the scene expressions, including camera movement, characters, dialogue, sound effects, etc., to obtain a scene sequence, such as: Shot 1: Long shot: Banners of the Shu army spread across the fields as they march south (pull-in shot) Shot 2, Medium Shot: Zhuge Liang waves his feather fan and orders the crossing of the Lu River (dialogue + gesture effects). Next, for each storyboard, its corresponding keyframe storyboard is generated (image generation). Then, based on the storyboard of each keyframe, the video is generated. This process requires the use of voice assets for lip-syncing and the addition of sound effects and background music to obtain the final video.

[0099] This disclosed work-driven video generation method belongs to the intersection of artificial intelligence and digital content creation, and specifically involves the following: The knowledge graph of the original work is automatically constructed; The automatic construction of the plot library corresponding to the atomic plot of the original work; Storyline assembly -> Automatic script generation; The character's digital assets are automatically generated or matched; Automatically create videos based on the character's digital assets and script, which are derivative works related to the original work.

[0100] This disclosure can semantically decompose any work to obtain a corresponding knowledge graph and plot library, thereby supporting any derivative work related to that work. It can generate a 3-5 minute animated video with consistent characters, synchronized audio and video, and ready for publication in a single step. This solution automates the entire pipeline from original work to derivative work, and it has at least the following advantages: 1. During the scriptwriting process, semantic and temporal consistency were fully considered to ensure that there were no temporal conflicts in the atomic plot sequence, and that the generated video was logically coherent and highly consistent with the content of the original work.

[0101] 2. Users only need to input script generation prompts, and the system can analyze the user's creative intent and automatically complete the secondary creation.

[0102] 3. Multimodal asset "dictionary" management: characters / scenes / voices are generated once and reused throughout the entire film, eliminating drifting.

[0103] Figure 7 This is a block diagram of a work-driven video generation apparatus according to an exemplary embodiment. The apparatus includes: The entity analysis module 710 is configured to perform entity analysis on the work and construct a knowledge graph corresponding to the work. The nodes in the knowledge graph are used to indicate the characters in the work, and the relationships between different nodes are used to indicate at least one of the following relationships between different characters: character relationship and event relationship. The plot library construction module 720 is configured to perform plot analysis on the work to obtain a plot library including multiple atomic plots; The prompt receiving module 730 is configured to execute a video generation prompt, wherein the video generation prompt is used to indicate the plot requirements corresponding to the video to be generated, and the plot requirements are related to the work. The script generation module 740 is configured to determine the script corresponding to the video based on the video generation prompts, the knowledge graph, and the multiple atomic plots. The video generation module 750 is configured to generate the video based on the script.

[0104] In one exemplary implementation, the script generation module 740 is configured to execute: In the knowledge graph, query the characters, character relationships, and event relationships related to the plot requirements to obtain the first query result; In the plot database, query the atomic plots related to the plot requirement to obtain a second query result; The script is constructed based on the first query result and the second query result.

[0105] In one exemplary embodiment, the events corresponding to the event relationships include corresponding first time information, each event corresponds to multiple atomic plots, the plot library is used to record each atomic plot, and the recorded atomic plot includes second time information; the script generation module 740 is configured to execute: If the first query result includes multiple event relationships, an event chain is obtained based on the temporal relationship indicated by the first time information of the corresponding multiple events; Based on the consistency between each atomic plot in the second query result and the event in the event chain, an atomic plot sequence is determined. The consistency includes time consistency determined based on the first time information and the second time information, and semantic consistency determined based on the semantics of the atomic plot and the semantics of the event. The script is constructed based on the atomic plot sequence, other information in the first query result, and the video generation prompts.

[0106] In one exemplary implementation, the plot library construction module 820 is configured to perform: Obtain atomic story templates corresponding to multiple story types. The atomic story templates are used to define the metadata corresponding to the story type. The metadata refers to the control attribute data required to present the atomic story of the corresponding story type. Based on the atomic plot template, each event in the work is split into plot segments, and the atomic plot template is filled based on the splitting results to obtain multiple atomic plots corresponding to the event; The multiple atomic plots are stored in the plot library.

[0107] In one exemplary implementation, the plot library construction module 820 is configured to perform: For any given atomic plot, extract the plot summary corresponding to that atomic plot; Feature extraction is performed on the plot summary to obtain the plot representation information corresponding to the atomic plot; Based on the plot summary, the atomic plot, and the plot representation information, construct the plot record corresponding to the atomic plot; The plot records are saved in the plot library.

[0108] In one exemplary implementation, the script generation module 740 is configured to execute: In the video generation prompt, extract the first prompt information used to indicate the requirements of the atomic plot; Extract the semantic representation information corresponding to the first prompt information; Based on the semantic representation information, a first number of target plot representation information are queried in the plot library, the first number being determined based on the video duration corresponding to the video. The second query result is obtained based on the atomic plots corresponding to each of the plot representation information.

[0109] In one exemplary implementation, the script generation module 740 is configured to execute: Based on the semantic consistency between each atomic plot and each event in the event chain, a second number of atomic plots are determined; The atomic plot sequence is determined based on the second time information of each of the second number of atomic plots.

[0110] In one exemplary embodiment, the video generation module 750 is configured to perform: For the entities in the script, extract the entity descriptions corresponding to the entities from the knowledge graph; Based on the entity description, determine the control attributes of the entity corresponding to multiple modalities; Based on the control attributes corresponding to the multiple modalities, determine the digital resources of the entity in the multiple modalities; The video is generated based on the entity's digital resources in multiple modalities.

[0111] In one exemplary embodiment, the video generation module 750 is configured to perform: Based on a large-scale generative language model, the script is semantically parsed and processed into scene sequences to obtain a scene sequence. The video is generated based on the digital resources of each entity in multiple modalities and the storyboard sequence.

[0112] Regarding the apparatus in the above embodiments, the specific manner of each step has been described in detail in the embodiments of the foregoing method, and will not be elaborated here.

[0113] Please refer to Figure 8 It illustrates the structural block of a computer device provided in an exemplary embodiment of this disclosure. Figure 1 The computer device can be a terminal. This computer device is used to implement the work-driven video generation method provided in the above embodiments. Specifically: Typically, computer device 800 includes a processor 801 and a memory 802.

[0114] Processor 801 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 801 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 801 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In an exemplary embodiment, processor 801 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In an exemplary embodiment, processor 801 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0115] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In an exemplary embodiment, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one instruction, at least one program, code set, or instruction set, configured to be executed by one or more processors to implement the above-described work-driven video generation method.

[0116] In one exemplary embodiment, the computer device 800 may optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 803 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 804, a touch display screen 805, a camera assembly 806, an audio circuit 807, a positioning assembly 808, and a power supply 809.

[0117] Those skilled in the art will understand that Figure 8The structure shown does not constitute a limitation on the computer device 800, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0118] Please refer to Figure 9 It illustrates the structural block of a computer device provided in another exemplary embodiment of this disclosure. Figure 2 The computer device can be a server for executing the aforementioned work-driven video generation method. Specifically: Computer device 900 includes a Central Processing Unit (CPU) 901, a system memory 904 including Random Access Memory (RAM) 902 and Read Only Memory (ROM) 903, and a system bus 905 connecting the system memory 904 and the CPU 901. Computer device 900 also includes a basic input / output system (I / O system) 906 that facilitates information transfer between various devices within the computer, and a mass storage device 907 for storing the operating system 913, application programs 914, and other program modules 911.

[0119] The basic input / output system 906 includes a display 908 for displaying information and an input device 909 for user input, such as a mouse or keyboard. Both the display 908 and the input device 909 are connected to the central processing unit 901 via an input / output controller 190 connected to the system bus 905. The basic input / output system 906 may also include the input / output controller 190 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 190 also provides output to a display screen, printer, or other types of output devices.

[0120] Mass storage device 907 is connected to central processing unit 901 via a mass storage controller (not shown) connected to system bus 905. Mass storage device 907 and its associated computer-readable media provide non-volatile storage for computer device 900. That is, mass storage device 907 may include computer-readable media (not shown) such as hard disk or CD-ROM (CompactDisc Read-Only Memory) drive.

[0121] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 904 and mass storage device 907 described above can be collectively referred to as memory.

[0122] According to various embodiments of this disclosure, the computer device 900 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 900 can be connected to a network 912 via a network interface unit 911 connected to a system bus 905, or the network interface unit 911 can be used to connect to other types of networks or remote computer systems (not shown).

[0123] The aforementioned memory also includes a computer program stored in the memory and configured to be executed by one or more processors to implement the aforementioned work-driven video generation method.

[0124] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set, when executed by a processor, implements the work-driven video generation method described above.

[0125] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0126] In an exemplary embodiment, a computer-readable storage medium including program code is also provided, such as a memory including program code that can be executed by a processor to perform the aforementioned work-driven video generation method. Optionally, the computer-readable storage medium may be read-only memory (ROM), random access memory (RAM), compact-disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0127] In an exemplary embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the above-described work-driven video generation method.

[0128] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0129] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A work-driven video generation method, characterized by, The method includes: By performing entity analysis on the work, a knowledge graph corresponding to the work is constructed. The nodes in the knowledge graph are used to indicate the characters in the work, and the relationships between different nodes are used to indicate at least one of the following relationships between different characters: character relationship and event relationship. The plot of the work was analyzed to obtain a plot library including multiple atomic plots; Receive a video generation prompt, the video generation prompt being used to indicate the plot requirements corresponding to the video to be generated, the plot requirements being related to the work; Based on the video generation prompts, the knowledge graph, and the multiple atomic plots, the script corresponding to the video is determined; The video is generated based on the script.

2. The method according to claim 1, characterized in that, The step of determining the script corresponding to the video based on the video-generated prompts, the knowledge graph, and the multiple atomic plots includes: In the knowledge graph, query the characters, character relationships, and event relationships related to the plot requirements to obtain the first query result; In the plot database, query the atomic plots related to the plot requirement to obtain a second query result; The script is constructed based on the first query result and the second query result.

3. The method according to claim 2, characterized in that, The events corresponding to the event relationships include corresponding first-time information. Each event corresponds to multiple atomic plots. The plot library is used to record each atomic plot, and the recorded atomic plot includes second-time information. The step of constructing the script based on the first query result and the second query result includes: If the first query result includes multiple event relationships, an event chain is obtained based on the temporal relationship indicated by the first time information of the corresponding multiple events; Based on the consistency between each atomic plot in the second query result and the event in the event chain, an atomic plot sequence is determined. The consistency includes time consistency determined based on the first time information and the second time information, and semantic consistency determined based on the semantics of the atomic plot and the semantics of the event. The script is constructed based on the atomic plot sequence, other information in the first query result, and the video generation prompts.

4. The method according to claim 2, characterized in that, The process of analyzing the plot of the work yields a plot library containing multiple atomic plots, including: Obtain atomic story templates corresponding to multiple story types. The atomic story templates are used to define the metadata corresponding to the story type. The metadata refers to the control attribute data required to present the atomic story of the corresponding story type. Based on the atomic plot template, each event in the work is split into plot segments, and the atomic plot template is filled based on the splitting results to obtain multiple atomic plots corresponding to the event; The multiple atomic plots are stored in the plot library.

5. The method according to claim 4, characterized in that, The step of storing the multiple atomic storylines in the storyline library includes: For any given atomic plot, extract the plot summary corresponding to that atomic plot; Feature extraction is performed on the plot summary to obtain the plot representation information corresponding to the atomic plot; Based on the plot summary, the atomic plot, and the plot representation information, construct the plot record corresponding to the atomic plot; The plot records are saved in the plot library.

6. The method according to claim 5, characterized in that, The process involves querying the plot database for atomic plots related to the plot requirement, yielding a second query result, including: In the video generation prompt, extract the first prompt information used to indicate the requirements of the atomic plot; Extract the semantic representation information corresponding to the first prompt information; Based on the semantic representation information, a first number of target plot representation information are queried in the plot library, the first number being determined based on the video duration corresponding to the video. The second query result is obtained based on the atomic plots corresponding to each of the plot representation information.

7. The method according to claim 3, characterized in that, The step of determining the atomic plot sequence based on the match between each atomic plot in the second query result and an event in the event chain includes: Based on the semantic consistency between each atomic plot and each event in the event chain, a second number of atomic plots are determined; The atomic plot sequence is determined based on the second time information of each of the second number of atomic plots.

8. The method according to claim 1, characterized in that, The process of generating the video based on the script includes: For the entities in the script, extract the entity descriptions corresponding to the entities from the knowledge graph; Based on the entity description, determine the control attributes of the entity in multiple modalities; Based on the control attributes corresponding to the multiple modalities, determine the digital resources of the entity in the multiple modalities; The video is generated based on the entity's digital resources in multiple modalities.

9. The method according to claim 8, characterized in that, The process of generating the video based on the entity's digital resources in multiple modalities includes: Based on a large-scale generative language model, the script is semantically parsed and processed into scene sequences to obtain a scene sequence. The video is generated based on the digital resources of each entity in multiple modalities and the storyboard sequence.

10. A work-driven video generation device, characterized in that, The device includes: The entity analysis module is configured to perform entity analysis on the work and construct a knowledge graph corresponding to the work. The nodes in the knowledge graph are used to indicate the characters in the work, and the relationships between different nodes are used to indicate at least one of the following relationships between different characters: character relationship and event relationship. The plot library construction module is configured to perform plot analysis on the work to obtain a plot library including multiple atomic plots; The prompt receiving module is configured to execute a video generation prompt, wherein the video generation prompt is used to indicate the plot requirements corresponding to the video to be generated, and the plot requirements are related to the work. The script generation module is configured to determine the script corresponding to the video based on the video generation prompts, the knowledge graph, and the multiple atomic plots; The video generation module is configured to generate the video based on the script.

11. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the work-driven video generation method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device performs the work-driven video generation method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes a computer program stored in a readable storage medium, wherein at least one processor of a computer device reads from and executes the computer program, causing the device to perform the work-driven video generation method as described in any one of claims 1 to 9.

Images