A shot generation method and system based on hierarchical narrative planning and multi-dimensional constraint feedback

Abstract

The application discloses a shot generation method and system based on hierarchical narrative planning and multi-dimensional constraint feedback, and belongs to the technical fields of computer vision, machine learning and natural language processing. By introducing a hierarchical narrative planning and multi-dimensional constraint feedback mechanism, the semantic deviation problem existing between a macro-narrative intention and a low-level shot composition in the related art is solved. By converting a narrative theme and an emotional arc into specific constraints, and combining narrative function labeling and film grammar constraints, the shot decomposition process can be internally associated with a narrative intention and professional specifications, so that a shot sequence meeting professional requirements in terms of narrative consistency and visual language standardization is generated.

Description

Technical Field

[0001] This application relates to the fields of computer vision, machine learning, and natural language processing, and in particular to a storyboard generation method and system based on hierarchical narrative planning and multidimensional constraint feedback. Background Technology

[0002] In the field of automatic storyboard generation for visual narrative content such as film, animation, etc., existing methods mainly adopt pipeline architecture based on sequence generation or multi-agent collaboration, which directly maps and generates shot sequences through semantic parsing of narrative text.

[0003] This approach has the following shortcomings: First, the planning process is conducted at a single, abstract level. There is a lack of a hierarchical constraint mechanism between the macro-level planning of narrative intent and the specific determination of shot parameters. This results in the higher-level narrative theme failing to effectively constrain the lower-level shot composition choices, leading to semantic discrepancies between the generated storyboard sequence and the original narrative intent. Second, film grammar rules are only used as external conditions for post-generation checks, rather than as internal constraint parameters in the shot decomposition process. This means that the determination of shooting parameters lacks a structured connection with narrative function annotations, making it difficult to guarantee the professional standardization of cinematic language.

[0004] The aforementioned shortcomings result in the automatically generated storyboard sequences failing to meet the quality requirements of professional storyboard creation in terms of both narrative consistency and visual language standardization, constituting a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] This application provides a storyboard generation method and system based on hierarchical narrative planning and multi-dimensional constraint feedback. The technical solution is as follows: On the one hand, a storyboard generation method based on hierarchical narrative planning and multi-dimensional constraint feedback is provided, the method comprising: Semantic parsing is performed on the input narrative text to obtain a set of narrative elements containing event sequences and character relationships; The narrative element set is planned at the first level to generate an emotional arc and narrative theme. Based on the preset refinement rules, the narrative theme is transformed into a set of theme keywords, and the emotional arc is transformed into emotional interval constraints. Using the set of thematic keywords and the emotional interval constraints as input constraints, a second-level planning is performed on the event sequence in the set of narrative elements to obtain the scene sequence and the narrative function annotation of each scene. The determination of the narrative function annotation is subject to the semantic association constraint of the set of thematic keywords. Based on the narrative function annotation, matching shooting specifications are retrieved from the preset film grammar constraint library. These specifications, along with the narrative intent parameters mapped by the theme keyword set, work together to decompose each scene in the scene sequence into shots, generating shooting parameters and image content descriptions for each shot, thus forming a storyboard sequence.

[0006] In one possible implementation, the step of semantically parsing the input narrative text to obtain a contextual feature representation includes: Based on the sentence boundaries of the input narrative text, the input narrative text is segmented to obtain a sequence of sentence-level text units; Encode each sentence-level text unit in the sentence-level text unit sequence to obtain the encoded representation of each sentence-level text unit; The encoded representations are concatenated according to the order of the sentence-level text units in the input narrative text to obtain the context feature representation.

[0007] In one possible implementation, identifying character entities, scene entities, and the event sequence in the input narrative text based on the contextual feature representation includes: Based on the contextual feature representation, identify character entities and scene entities in the input narrative text; Based on the contextual feature representation, events in the input narrative text are identified to obtain an initial event set; The initial set of events is arranged according to the order in which each event appears in the input narrative text to obtain the event sequence.

[0008] In one possible implementation, determining the role relationship based on the event sequence and the role entity includes: Extract interaction events between the role entities from the event sequence; Based on the interaction events, determine the type and strength of the relationship between the role entities; The role relationship is determined based on the type and strength of the role relationship.

[0009] In one possible implementation, determining the event influence of each event based on the causal influence range of each event in the event sequence includes: Based on the aforementioned causal impact range, determine the set of events that directly affect each event and the set of events that are affected. The causal propagation depth of each event is determined based on the number of events in the directly affecting event set and the affected event set. The event impact of each event is determined based on the causal propagation depth.

[0010] In one possible implementation, generating the narrative theme based on the key events includes: Based on the key events, determine the core narrative theme of the event sequence; Based on the core narrative theme and the distribution of event types in the event sequence, the emotional tone of the narrative theme is determined; The core narrative theme and the emotional tone are combined to form the narrative purpose.

[0011] In one possible implementation, the step of extracting keywords from the narrative theme according to the preset refinement rules to obtain the theme keyword set includes: Based on the preset refinement rules, the narrative theme is semantically decomposed to obtain a sequence of semantic units; From the sequence of semantic units, select semantic units that represent the core narrative intent; The selected semantic units are identified as the thematic keywords in the set of thematic keywords.

[0012] In one possible implementation, converting the emotional target values ​​of each stage in the emotional arc into corresponding numerical ranges includes: Determine the emotional intensity level of the emotional target value at each stage of the emotional arc; Based on the emotional intensity level, determine the upper and lower limits of the numerical range corresponding to each stage; The upper limit value and the lower limit value are combined to form the numerical range.

[0013] In one possible implementation, combining the numerical intervals in chronological order to obtain the emotional interval constraint includes: Determine the time position of each of the numerical intervals in the emotional arc; Based on the stated time position, the numerical intervals are arranged in chronological order; The arranged numerical intervals are encapsulated as the emotional interval constraints.

[0014] In one possible implementation, the step of dividing the event sequence into scenes based on the event boundaries of each event in the event sequence and the set of narrative elements to obtain the scene sequence, wherein the boundaries of the scene division do not truncate the causal chain of events in the event sequence, includes: Based on the connection relationship between adjacent events in the event sequence, the event boundary of each event is determined; Based on the event boundaries and the set of narrative elements, the event sequence is segmented to obtain candidate scenes; Perform causal chain integrity verification on the boundaries of the candidate scenarios; The candidate scene boundary of the truncated causal chain is adjusted to the endpoint of the causal chain to obtain the scene sequence.

[0015] In one possible implementation, performing causal chain integrity verification on the boundaries of the candidate scenarios includes: Determine whether there are cross-boundary causal pairs at the boundaries of each candidate scene; Based on the candidate scenarios where the causal events and the result events are located in the causal relationship, determine whether the causal chain is truncated by the boundary of the candidate scenario; The judgment result is marked as the verification status of the candidate scenario.

[0016] In one possible implementation, assigning an emotion change target to each scene in the scene sequence, using the emotion interval constraint as an input constraint, includes: Based on the aforementioned emotional interval constraints, the target range of emotional changes for each scene in the scene sequence is determined; Assign initial emotion change targets to each scenario from the range of emotion change targets; Based on the initial emotional change targets of adjacent scenes in the scene sequence, the initial emotional change targets of each scene are adjusted to obtain the emotional change targets.

[0017] In one possible implementation, adjusting the initial emotion change target for each scene based on the initial emotion change targets of adjacent scenes in the scene sequence to obtain the emotion change target includes: Determine the progressive difference of the initial emotional change target between adjacent scenes in the scene sequence; Based on the magnitude of the progressive difference and the target range of the emotional change, determine the scenarios that need adjustment; The initial emotional change target of the scene that needs adjustment is adjusted to the boundary value of the range of the emotional change target to obtain the emotional change target.

[0018] In one possible implementation, determining the narrative function annotation of each scene based on the events contained in each scene and the set of topic keywords includes: Based on the events contained in each scene, determine the candidate set of narrative functions for each scene; Determine the semantic relevance between each scenario and each topic keyword in the aforementioned topic keyword set; Based on the semantic relevance, the narrative function label for each scene is determined from the narrative function candidate set, so that the narrative function label for each scene is semantically associated with at least one topic keyword in the topic keyword set.

[0019] In one possible implementation, determining the shot size, shooting angle, and camera movement for each shot in the shot sequence as shooting parameters includes: Based on the shooting specifications, a set of candidate shooting parameters is determined for each shot, which includes candidate shot size, candidate shooting angle, and candidate shot movement mode; Based on the narrative intent parameters, the initial shooting parameters for each shot are selected from the set of candidate shooting parameters; Based on the constraint of the difference in framing between adjacent shots, the framing in the initial shooting parameters of each shot is corrected to obtain the shooting parameters.

[0020] In one possible implementation, generating a scene content description for each shot includes: Based on the shooting parameters of each shot, determine the composition framework of each shot; Based on the aforementioned composition framework, determine the set of image elements for each shot; Based on the set of image elements, generate the image content description for each shot. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a storyboard generation method based on hierarchical narrative planning and multidimensional constraint feedback provided in an embodiment of this application; Figure 2 This is a flowchart of another storyboard generation method based on hierarchical narrative planning and multidimensional constraint feedback provided in the embodiments of this application; Figure 3 This is a flowchart of another storyboard generation method based on hierarchical narrative planning and multidimensional constraint feedback provided in the embodiments of this application; Figure 4 This is a flowchart of another storyboard generation method based on hierarchical narrative planning and multidimensional constraint feedback provided in the embodiments of this application; Figure 5This is a flowchart of another storyboard generation method based on hierarchical narrative planning and multidimensional constraint feedback provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0024] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor are there any restrictions on quantity or execution order.

[0025] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions. Automatic storyboard generation methods for visual narrative content in film, animation, and other fields primarily generate shot sequences by directly mapping semantic analysis of the narrative text. However, these methods suffer from a single level of abstraction during the planning process, making it difficult for the macro-narrative intent to effectively constrain low-level shot composition choices. This results in semantic discrepancies between the generated storyboard sequence and the original narrative intent. Furthermore, film grammar rules are often used as external checks after generation, rather than as internal constraints during shot decomposition. This lack of structured connection between the determination of shooting parameters and narrative function annotation makes it difficult to guarantee the professional standardization of cinematic language. These shortcomings lead to automatically generated storyboard sequences failing to meet the quality requirements of professional storyboard creation in both narrative consistency and visual language standardization.

[0026] To address this, this application proposes a storyboard generation method based on hierarchical narrative planning and multidimensional constraint feedback, such as... Figure 1 As shown, taking the generation system (hereinafter referred to as the system) as the execution subject as an example, the method includes: 101. Perform semantic parsing on the input narrative text to obtain a set of narrative elements containing event sequences and character relationships; 102. Perform first-level planning on the set of narrative elements to generate an emotional arc and narrative theme. Based on the preset refinement rules, transform the narrative theme into a set of theme keywords and the emotional arc into emotional range constraints. 103. Using the set of keywords for the theme and the emotional interval constraint as input constraints, perform second-level planning on the event sequence in the set of narrative elements to obtain the scene sequence and the narrative function label of each scene. The determination of the narrative function label is constrained by the semantic association of the set of keywords for the theme. 104. Based on the narrative function annotation, the matching shooting specifications are retrieved from the preset film grammar constraint library. Together with the narrative intent parameters mapped by the theme keyword set, the scenes in the scene sequence are decomposed into shots, and the shooting parameters and screen content descriptions of each shot are generated to form a storyboard sequence.

[0027] For ease of understanding, the following explains some key terms in this embodiment: Narrative text refers to the original text content that serves as the input source for storyboard generation, such as novels, scripts, or story outlines.

[0028] Semantic parsing refers to the natural language processing of narrative texts to extract the deep semantic information contained within them.

[0029] The narrative element set refers to the core components extracted from a narrative text through semantic analysis, including event sequences and character relationships. An event sequence refers to a series of actions or state changes that occur in chronological order within the story. Character relationships refer to the interaction patterns, emotional connections, or social statuses that exist between different characters in the story.

[0030] The first level of planning refers to the macro-level narrative structure design of the collection of narrative elements, which mainly focuses on the overall direction of the story and the expression of its core ideas.

[0031] The emotional arc refers to the overall trend of emotional changes in a story, which is usually presented in the form of an undulating curve.

[0032] The narrative theme refers to the core idea, theme, or deeper meaning that the story aims to convey.

[0033] Preset refinement rules refer to a set of rules used to transform high-level narrative concepts into specific, actionable constraints. For example, this set of preset refinement rules includes at least the following rules: keyword extraction rules, whose input is the narrative theme text and output is a set of thematic keywords. Intermediate processing steps include word segmentation, part-of-speech tagging, and importance calculation of the narrative theme, and selecting a preset number of words as thematic keywords according to their importance from high to low; semantic unit decomposition rules, whose input is the narrative theme text and output is a sequence of semantic units. Intermediate processing steps include dependency parsing or semantic role tagging of the narrative theme to identify subject-verb-object structures, modification relationships, and core concepts; and semantic unit filtering rules, whose input is a sequence of semantic units and output is the filtered semantic units. Intermediate processing steps include calculating the importance score of each semantic unit and selecting those above a preset threshold. The system comprises: a semantic unit; an emotion intensity level rating rule, whose input is the emotion target value for each emotion stage and output is the emotion intensity level for each emotion stage; intermediate processing steps including comparing the emotion target value with a preset emotion intensity level scale to determine its level; a numerical interval mapping rule, whose input is the emotion target value and its emotion intensity level for each emotion stage and output is the numerical interval corresponding to each emotion stage; intermediate processing steps including determining the half-width based on the emotion target value and the emotion intensity level, and calculating the lower and upper limits; and a time sequence combination rule, whose input is each numerical interval and its time position in the emotion arc, and output is the emotion interval constraint; intermediate processing steps including determining the time position of each numerical interval and arranging and encapsulating them in chronological order.

[0034] A set of thematic keywords refers to a combination of words extracted from the narrative theme that represents the core theme of the story.

[0035] Emotional range constraints refer to the numerical range restrictions set on the intensity or type of emotions at different stages of a story.

[0036] The second level of planning refers to organizing the event sequence into a specific scene sequence at the meso level and assigning narrative function labels to each scene.

[0037] A scene sequence refers to a series of scenes arranged in chronological order within a story.

[0038] Narrative function labeling refers to tags that describe the specific narrative role or purpose of each scene in the story.

[0039] The preset film grammar constraint library refers to a knowledge base that stores professional standards for commonly used camera language, composition, and editing in filmmaking.

[0040] Shooting guidelines refer to the specific rules retrieved from a pre-defined library of film grammar constraints to guide the shooting of shots.

[0041] Narrative intent parameters are parameters derived from the set of thematic keywords and used to guide shot composition and expressiveness.

[0042] Shot breakdown refers to breaking down a complete scene into multiple consecutive shots.

[0043] Shooting parameters refer to the technical details that describe each shot, such as shot size, shooting angle, and camera movement.

[0044] The description of the content of the shot refers to the description of the visual elements and composition layout contained in each shot.

[0045] A storyboard sequence is a collection of shots arranged in chronological order, containing shooting parameters and descriptions of the content of each shot.

[0046] Contextual feature representation refers to the numerical representation obtained after encoding the input narrative text, which can characterize the semantic information and contextual relationships of the text.

[0047] A key event is an event that has a decisive impact on the development of the narrative.

[0048] A causal chain is a chain of events that are interconnected by cause and effect.

[0049] Emotional change targets refer to indicators assigned to a scene to represent the degree of change in the intensity or tendency of emotions in that scene.

[0050] A preset threshold refers to a pre-set numerical limit used for judgment.

[0051] A lens unit refers to a structured data unit formed by associating the lens's shooting parameters with the description of the image content.

[0052] This application provides a storyboard generation method based on hierarchical narrative planning and multidimensional constraint feedback. This method performs semantic parsing on the input narrative text to obtain a set of narrative elements containing event sequences and character relationships. Specifically, a rule-based approach can be used, employing pre-defined lexical and syntactic rules to identify noun phrases as potential characters or events from the text, and determining the relationships between them based on verb phrases. For example, a series of pattern matching rules can be manually defined; when the sentence structure "A did B" appears in the text, A is identified as a character, B as an event, and an action relationship between A and B is established. This approach can initially extract core information from the narrative text, laying the foundation for subsequent planning.

[0053] The aforementioned multidimensional constraints include: narrative semantic constraints represented by the set of thematic keywords, emotional rhythm constraints represented by the emotional interval constraints, narrative logic constraints represented by the semantic association between the narrative function annotation and the set of thematic keywords, and film grammar constraints represented by the shooting specifications. The constraints of each dimension are injected step by step as input parameters of the corresponding hierarchical planning process according to the hierarchical structure of the hierarchical narrative planning, rather than being verified as external check conditions after the planning is completed.

[0054] The first level of planning for this set of narrative elements generates an emotional arc and a narrative theme. Based on pre-defined refinement rules, the narrative theme is transformed into a set of thematic keywords, and the emotional arc is transformed into emotional range constraints. For example, an expert can subjectively judge the overall emotional trend of the story based on the overall sequence of events and draw a rough emotional arc. Simultaneously, the expert can extract several core words as the narrative theme based on their understanding of the story content and directly use them as the set of thematic keywords. Emotional range constraints simply map the key points on the emotional arc to fixed numerical ranges. This planning method allows for a high-level grasp of the story's overall structure and emotional tone.

[0055] Using the set of thematic keywords and the emotional range constraint as input constraints, a second-level planning process is performed on the event sequence within the narrative element set to obtain a scene sequence and a narrative function label for each scene. The determination of this narrative function label is constrained by the semantic association of the set of thematic keywords. Specifically, the event sequence can be simply divided according to a fixed time length or number of events to form a preliminary scene sequence. Each scene is assigned a pre-defined narrative function label, such as "opening," "development," "climax," or "ending," ensuring that this label has a direct lexical match with a word in the set of thematic keywords. For example, if the thematic keyword contains "conflict," then scenes containing fighting events can be labeled as "conflict scenes." This planning method aims to refine the macro-narrative intent to the scene level.

[0056] Based on the narrative function annotation, matching shooting specifications are retrieved from a pre-defined film grammar constraint library. These specifications, along with the narrative intent parameters mapped to the theme keyword set, work together to decompose each scene in the scene sequence into shots, generating shooting parameters and image content descriptions for each shot, thus forming a storyboard sequence. For example, a fixed set of shooting specifications can be preset for each narrative function annotation. When a scene is labeled "opening," parameters such as long shot, level shot, and fixed shot are uniformly used. The narrative intent parameters can simply map theme keywords to shot composition tendencies; for example, "grand" maps to a wide-angle shot. Each scene is decomposed into a fixed number of shots, and each shot is assigned preset shooting parameters and image content description templates. For example, a "dialogue" scene can be decomposed into two shots, a close-up A and a close-up B, with image content descriptions of "character A's facial close-up" and "character B's facial close-up." In this way, the initial transformation from scene to shot can be achieved, forming a complete storyboard sequence.

[0057] This application addresses the semantic discrepancy between macro-narrative intent and low-level shot composition in related technologies by introducing hierarchical narrative planning and a multi-dimensional constraint feedback mechanism. By transforming the narrative theme and emotional arc into specific constraints, and combining them with narrative function annotations and film grammar constraints, the shot decomposition process can be intrinsically linked to narrative intent and professional standards, thereby generating a storyboard sequence that meets professional requirements in terms of both narrative consistency and visual language standardization.

[0058] In the storyboard generation process, semantic analysis of the input narrative text is fundamental to constructing the narrative element set. However, if the semantic analysis process fails to fully capture the deeper meaning and structure of the text—for example, failing to effectively identify the contextual features of the text, accurately distinguish various entities, or clearly define the relationships between events—the resulting narrative element set may be incomplete or inaccurate. This directly affects the quality of subsequent layered planning, resulting in a lack of solid data support for aspects such as the extraction of emotional arcs and narrative themes, scene division, and shot decomposition, thereby impacting the narrative coherence and expressiveness of the storyboard sequence.

[0059] To address this, this application further proposes semantic parsing of the input narrative text to obtain a set of narrative elements containing event sequences and character relationships, see [link to relevant documentation]. Figure 2 The process includes the following steps: 201. Perform semantic parsing on the input narrative text to obtain contextual feature representations.

[0060] Contextual feature representation is an abstract form of text that captures its semantic information and contextual relationships after processing. Its role is to provide a unified and rich semantic foundation for subsequent entity recognition and event sequence extraction. In practical implementation, various Natural Language Processing (NLP) techniques can be employed. For example, deep learning-based models, such as Transformer, BERT, and GPT, can be used to encode the input narrative text and generate high-dimensional vector representations. These vectors can capture semantic information at the word, sentence, and even document levels. Another approach is to extract the syntactic structure and word dependencies of the text using traditional NLP methods such as syntactic analysis and dependency parsing, and integrate them into contextual feature representations.

[0061] 202. Based on the contextual features, identify the character entities, scene entities, and event sequence in the input narrative text.

[0062] After obtaining the contextual feature representation of the text, this application further identifies character entities, scene entities, and event sequences in the input narrative text based on this contextual feature representation. Character entities refer to characters or subjects with action capabilities in the story; scene entities refer to the time, place, or environment in which the story takes place; and event sequences are a series of actions or state changes that occur in chronological order within the story. Identifying these entities and sequences is a crucial step in constructing the narrative skeleton. For example, sequence labeling-based models (such as Bi-LSTM-CRF and Transformer-CRF) can be used to process the contextual feature representation to identify named entities (including character names, location names, and times) in the text as character entities and scene entities. For the identification of event sequences, event extraction techniques can be used. By identifying event trigger words and event arguments, and combining them with time information, actions or state changes in the text can be extracted and arranged according to their order of appearance or logical order in the text to form an event sequence.

[0063] 203. Determine the relationship between roles based on the event sequence and the role entities.

[0064] To gain a more comprehensive understanding of the narrative, this application also determines character relationships based on the identified event sequences and character entities. Character relationships refer to the interactions, emotional, social, or causal connections between different character entities in a story. Clarifying character relationships helps in understanding the story's conflict, development, and character motivations, providing a deeper understanding for the subsequent emotional arcs and narrative themes. In practice, relationships can be determined by analyzing the interactions between character entities in an event sequence. For example, if two character entities frequently appear as participants in the same event, or if one character performs an action on another, a relationship can be inferred. Relationship types can include kinship, friendship, antagonism, cooperation, etc. Relationship strength can be quantified using indicators such as interaction frequency and interaction nature (positive / negative). This can be achieved using relationship extraction models (such as relationship classification models based on graph neural networks or Transformers), taking an event sequence and character entities as input, and outputting the relationship type and strength between the character pairs.

[0065] 204. The relationship between the character entity, the scene entity, the event sequence, and the character is determined as the narrative element set.

[0066] In this application, the identified character entities, scene entities, event sequences, and character relationships are integrated to form a narrative element set. This narrative element set is the foundational data structure for the storyboard generation method. It structurally integrates all key information extracted from the original narrative text, providing comprehensive and accurate input for subsequent layered planning. This step is a data integration and encapsulation process, uniformly storing the previously identified list of character entities, list of scene entities, event sequences (usually event objects arranged chronologically, each event object containing information such as event type, participating characters, and time / location of occurrence), and character relationship graph (or relationship list) in a single data structure, such as a JSON object, XML file, or database record.

[0067] By employing the aforementioned technical solution, the contextual feature representation of the input narrative text is obtained during semantic parsing, providing a rich semantic foundation for subsequent entity and event recognition. This enables more accurate identification of character entities, scene entities, and event sequences within the text, avoiding element omissions or errors due to insufficient semantic understanding. By analyzing the interactions between event sequences and character entities, character relationships are clarified, ensuring that the narrative element set not only includes static entities and events but also incorporates dynamic interactive information, enriching the depth and breadth of the narrative content. Integrating this refined and structured information into a narrative element set ensures its comprehensiveness, accuracy, and internal logic, providing high-quality input for subsequent first- and second-level planning, thereby effectively improving the narrative coherence and expressiveness of the storyboard generation results.

[0068] In some of the embodiments described above in this application, semantic parsing of the input narrative text is proposed to obtain contextual feature representations. However, when performing semantic parsing directly on the entire input narrative text, problems such as excessive text length, difficulty in accurately capturing semantic information, and high computational resource consumption may be encountered, which may lead to a decrease in the accuracy of subsequent character entity, scene entity, and event sequence recognition.

[0069] To address this, this application further proposes semantic parsing of the input narrative text to obtain a contextual feature representation, including: segmenting the input narrative text according to the sentence boundaries to obtain a sequence of sentence-level text units; encoding each sentence-level text unit in the sequence to obtain an encoded representation of each sentence-level text unit; and concatenating the encoded representations according to the order of the sentence-level text units in the input narrative text to obtain the contextual feature representation.

[0070] This step aims to decompose the original input narrative text into a series of independent, sequentially arranged sentence-level text units. Its core lies in utilizing inherent sentence boundary information within the text, such as punctuation marks like periods, question marks, and exclamation marks, and combining this with linguistic rules (such as handling abbreviations and ellipses) to precisely segment the continuous text stream into semantically complete sentences. This approach effectively reduces the complexity of subsequent processing and ensures that each processing unit has a clear semantic boundary. Segmentation can be performed using a rule-based parser that pre-defines sentence termination rules and special structure handling rules for various languages; alternatively, machine learning-based sequence labeling models (such as Conditional Random Fields (CRF), Bi-LSTM-CRF combined with Bi-LSTM, Transformer-based sequence labeling models, etc.) can be used to train and identify sentence boundary markers in the text, thereby achieving more robust sentence segmentation.

[0071] The purpose of this step is to convert each sentence-level text unit into a machine-understandable numerical vector form, i.e., its encoded representation. This encoded representation captures the semantic, syntactic, and contextual information of the sentence, providing a foundation for subsequent semantic analysis. By converting text into points in a high-dimensional vector space, similarity calculations, clustering, and other operations can be easily performed. Various text encoding techniques can be employed. For example, pre-trained word embedding models (such as Word2Vec and GloVe) can be used to convert each word in a sentence into a vector, and then the word vector sequence can be aggregated through averaging, weighted averaging, or using recurrent neural networks (such as LSTM and GRU) to obtain a sentence-level encoded representation. More advanced methods include using pre-trained language models based on the Transformer architecture (such as BERT, RoBERTa, and Sentence-BERT), which can generate sentence embeddings containing rich contextual information. Their encoded representations are typically vectors of specific layers or specific tokens (such as [CLS] tokens) output by the model.

[0072] This step aims to combine the encoded representations of the individual sentence-level text units obtained in the previous step according to their natural chronological order within the original input narrative text, thereby constructing a contextual feature representation that reflects the overall semantics and contextual relationships of the entire narrative text. This concatenation operation ensures that the coherence and logical structure of the narrative text are preserved in the numerical representation, allowing subsequent semantic parsing to fully utilize the overall context of the text. The concatenation can be achieved by simply linking all sentence encoded representations end-to-end to form a longer vector. Alternatively, to better capture long-distance dependencies and compress information between sentences, a sequence processing model (such as another Transformer encoder or a Long Short-Term Memory network LSTM) can be used before or after concatenation to further process these sentence encoded representation sequences, generating a more compact and context-rich contextual feature representation. This approach effectively integrates the semantic relationships between different sentences, forming a unified and comprehensive text feature vector.

[0073] The above technical solution segments the input narrative text based on sentence boundaries, obtaining a sequence of sentence-level text units. This solves the problems of semantic information loss and low computational efficiency that may occur when directly processing long texts. Each sentence-level text unit is encoded to generate its independent semantic representation, ensuring accurate capture of local semantics. These encoded representations are then concatenated according to the order of the sentence-level text units within the input narrative text, constructing a contextual feature representation that contains both fine-grained semantic information and preserves the overall narrative context. This hierarchical processing method improves the accuracy and efficiency of semantic parsing, providing high-quality input for subsequent identification of character entities, scene entities, and event sequences, thereby ensuring the robustness and reliability of the entire storyboard generation method.

[0074] The baseline method in this application involves semantic parsing of the input narrative text to obtain a set of narrative elements containing event sequences and character relationships. Although this method can obtain contextual feature representations, accurately and systematically identifying all character and scene entities directly from these raw features and constructing a logically clear event sequence may face challenges when dealing with complex or lengthy narrative texts. In particular, during event identification, without refined processing, the boundaries, types, and accurate ordering of events on the narrative timeline may become blurred or inaccurate, thus affecting the accuracy of subsequent narrative planning and the quality of storyboard generation.

[0075] To address this, this application further proposes a step of identifying character entities, scene entities, and the event sequence in the input narrative text based on the context feature representation. This step specifically includes: identifying character entities and scene entities in the input narrative text based on the context feature representation; identifying events in the input narrative text based on the context feature representation to obtain an initial event set; and arranging the initial event set according to the order in which each event appears in the input narrative text to obtain the event sequence.

[0076] When identifying character and scene entities in an input narrative text based on its contextual feature representation, the aim is to accurately extract all key character and scene entities mentioned in the text from the semantically parsed contextual feature representation. Character entities typically refer to people, animals, or non-living things with personality traits in the story, while scene entities refer to the time, place, or environmental elements in which the story takes place. Named Entity Recognition (NER) technology can be used, combined with a pre-trained language model (e.g., a Transformer-based model) for fine-tuning, to identify names of people, places, organizations, etc., in the text. For example, the model can learn patterns related to entity types in the contextual feature representation and label words or phrases in the text as character or scene entities using sequence labeling (such as BIOES labeling). Furthermore, rule-based methods can be used, employing predefined entity dictionaries and pattern matching to assist in identification and verification.

[0077] When identifying events in an input narrative text based on its contextual feature representation to obtain an initial event set, the goal is to identify specific events occurring within the narrative text from the contextual feature representation and aggregate them into an initial event set. Events are a core component of narrative text, typically containing event trigger words, event arguments (such as participants, time, and location), and event type. Event extraction techniques can be employed. For example, deep learning models (such as Transformer-based sequence labeling models or graph neural networks) can be used to analyze the contextual feature representation, identify event trigger words, and further identify the argument roles associated with those trigger words. By learning from a large amount of event-labeled data, the model can understand the likelihood of different words acting as event trigger words in a specific context and identify the boundaries and types of events. Each identified event can be represented as a structured data unit containing information such as event type, trigger words, arguments, and their roles.

[0078] Based on the order in which each event appears in the input narrative text, the initial set of events is arranged to obtain the event sequence. This step aims to logically arrange the initial set of events identified in the previous step, which may have been out of order or not explicitly ordered, according to their actual chronological order of occurrence in the original input narrative text, thus forming an ordered event sequence. The event sequence is the foundation for subsequent narrative planning, and its accuracy directly affects the coherence and logic of the story. The most direct method is to record the starting position or the position of the main trigger word of each event in the original text. After obtaining the initial set of events, the events are sorted in ascending order based on this positional information. For events spanning multiple sentences or complex events, more advanced temporal relationship extraction techniques can be used to help determine a more precise relative order by analyzing temporal connectors (such as "after," "at the same time," "before") or timestamp information between events.

[0079] The aforementioned technical solutions enable more accurate and systematic identification of key elements in the input narrative text during semantic analysis. By directly identifying character and scene entities from contextual feature representations, the comprehensiveness and accuracy of entity information are ensured, laying a solid foundation for subsequent character relationship determination and scene segmentation. Separating event identification from event sequence construction—first identifying all events to form an initial set, then arranging them according to their natural order of appearance in the text—effectively avoids the problem of disordered order caused by the complexity of the event identification process. This step-by-step processing method makes the generation of event sequences more logically clear and structurally rigorous, ensuring that the obtained event sequences accurately reflect the timeline and story development of the original narrative text. This not only improves the quality of the narrative element set but also provides high-quality input for subsequent first- and second-level narrative planning, thereby enhancing the overall accuracy and reliability of the storyboard generation method.

[0080] In some of the embodiments described above in this application, the relationship between characters is determined based on the sequence of events and the characters themselves. However, in practical applications, if the relationship between characters is determined only in a general way based on the sequence of events and the characters themselves, it may be difficult to accurately capture the complex and dynamic interaction patterns between characters and their deeper meanings, thereby affecting the accuracy of subsequent narrative planning and the precision of storyboard generation.

[0081] In response, this application further proposes a method for determining role relationships based on event sequences and role entities, which includes: extracting interaction events between role entities from the event sequence; determining the role relationship type and relationship strength between role entities based on the interaction events; and determining the role relationship based on the role relationship type and relationship strength.

[0082] For example, extracting interaction events between character entities from an event sequence refers to identifying events in a narrative text where two or more character entities directly or indirectly interact, communicate, or influence each other. This can be achieved using Natural Language Processing (NLP) techniques, such as dependency parsing, semantic role labeling, or event extraction models. An event pattern library can be built, containing common interaction verbs (such as "talk," "help," "confront," "cooperate," etc.) and their corresponding semantic roles (agent, patient, object, etc.). When an event in the event sequence matches these patterns, and its semantic roles contain two or more identified character entities, the event is identified as an interaction event. For example, in the event "Zhang San helped Li Si," "help" is the interaction verb, and "Zhang San" and "Li Si" are character entities; this event is thus extracted as an interaction event.

[0083] Based on the interaction event, the type and strength of the relationship between the roles are determined. The type of relationship refers to the specific nature of the relationship between roles, such as friend, enemy, relative, collaborator, etc.; the strength of the relationship quantifies the closeness or importance of this relationship. For determining the relationship type, a pre-defined ontology of role relationship types can be used, containing various common role relationships and their semantic features. For each extracted interaction event, machine learning models (such as Support Vector Machine (SVM), Naive Bayes, Random Forest, etc., text classifiers) or rule-based reasoning systems can be used to analyze the verbs, objects, modifiers, and the context of the event, mapping them to the pre-defined relationship type. For example, "Zhang San helped Li Si" might point to a "friend" or "collaborator" relationship; "Zhang San and Li Si had a dispute" might point to an "enemy" or "competitor" relationship. The strength of the relationship can be quantified in various ways. For example, the frequency of specific interactive events in an event sequence can be counted; different weights can be assigned to interactive events based on their semantic importance (e.g., interactive events in key plot points have higher weights); and sentiment analysis can be used to assess the emotional tendencies and intensity between characters in interactive events. These quantitative indicators can be accumulated or weighted averaged to form a comprehensive relationship strength value. For example, frequent "help" events increase the strength of the "friend" relationship.

[0084] Based on the type and strength of the relationship, the character relationship is determined. This step integrates the previously identified relationship types and strengths to form a character relationship description. All identified relationship types and their corresponding strengths between each pair of character entities can be summarized. If multiple relationship types exist, they can be sorted by strength or combined into a multi-dimensional relationship vector. For example, character A and character B might have both a "collaborator" relationship (strength 0.8) and a "competitor" relationship (strength 0.3). The determined character relationship can be structured data, such as a tuple (character A, character B, list of relationship types, list of relationship strengths), or edges and their attributes in a relationship graph. This determined character relationship will be used as part of the narrative element set for subsequent narrative planning.

[0085] The aforementioned technical solution enables precise identification of specific interaction events between character entities from an event sequence, allowing for detailed analysis of these events and clarifying the specific types and strengths of character relationships. This fine-grained approach to character relationship identification overcomes the ambiguity of merely identifying general relationships, leading to a deeper and more accurate understanding of the complex dynamics between characters in the narrative text. Consequently, subsequent first-level planning, when generating emotional arcs and narrative themes, can more fully consider the impact of character interactions on narrative direction and emotional changes, thus generating more persuasive and impactful narrative themes and emotional arcs. Furthermore, precise character relationship information provides a solid foundation for scene division and narrative function annotation in second-level planning, ensuring that the storyboard sequence can more accurately capture and express the deep connections between characters during shot decomposition, generating storyboard sequences that better align with narrative intent and emotional expression.

[0086] This application further proposes that, when planning the first level of the narrative element set to generate the emotional arc and narrative theme, if there is a lack of effective extraction of the core content of the narrative and accurate grasp of the emotional direction, the generated emotional arc and narrative theme may not be consistent with the deeper meaning of the original narrative text, thereby affecting the accuracy and artistic expression of the subsequent storyboard generation.

[0087] To address the aforementioned issues, this application proposes a first-level planning approach for the narrative element set to generate an emotional arc and narrative theme. (See [link to relevant documentation]). Figure 3 ,include: 301. Identify key events from this event sequence; 302. Based on this key event, generate the narrative theme; 303. Generate the emotional arc based on the event type distribution of each event in the event sequence.

[0088] For example, the step of identifying key events from a sequence of events aims to sift through the complete sequence of events in the story to find those events that have a decisive impact on the narrative development. These key events are often core elements that drive the plot forward, trigger major conflicts, or mark important turning points. Identifying key events can be achieved in several ways. For instance, the strength of causal relationships between events can be analyzed, marking those that are the cause or consequence of numerous subsequent events as key; alternatively, the frequency of an event's mention in the narrative text or its role in changes in character relationships can be assessed, with high-frequency or strongly correlated events being more likely to be key events. Furthermore, pre-trained natural language processing models can be used to automatically identify key events by analyzing their semantic importance or their position in the story structure (such as climaxes or turning points).

[0089] The step of generating the narrative theme based on the key event aims to extract the deep theme or core idea of ​​the story from the identified core event. The narrative theme is the fundamental meaning, value, or lesson the story conveys. Generating the narrative theme can be based on semantic analysis of the key event, for example, by extracting recurring keywords, concepts, or conflict patterns from the key event and summarizing them into one or a few concise statements. Alternatively, it can be achieved by constructing logical relationships between key events to deduce the message the story conveys or the growth and transformation experienced by the characters, thus forming the narrative theme. For example, if the key events revolve around courage and sacrifice, the narrative theme might be related to "selfless dedication" or "overcoming fear."

[0090] The core of generating the emotional arc based on the distribution of event types within the event sequence lies in depicting the emotional trajectory of the narrative by analyzing the frequency and order of different event types. The emotional arc reflects the changing emotional tone and intensity of the story at different stages. This can be achieved by pre-defining a series of event types (such as positive events, negative events, conflict events, resolution events, suspense events, etc.) and then labeling each event in the event sequence with its type. Statistical analysis of the distribution of these event types along the timeline is then performed; for example, a series of positive events may indicate an emotional rise, while a conflict event followed by a negative event may indicate an emotional trough. Through this distribution pattern, a curve reflecting the emotional trajectory of the story—the emotional arc—can be constructed, visually showcasing the tension, relaxation, climax, and trough of the story.

[0091] The aforementioned technical solution enables first-level planning of the narrative element set in a structured and data-driven manner. By accurately identifying key events from the event sequence, it ensures that the generated narrative theme profoundly reflects the core content and deeper meaning of the story, avoiding generalization or deviation. Simultaneously, generating emotional arcs based on the event type distribution of each event in the sequence makes the depiction of emotional trajectory more objective and accurate, realistically reproducing the emotional fluctuations of the story and providing a solid emotional foundation for subsequent scene planning and shot breakdown. This precise grasp of the core narrative content and emotional trajectory enhances the quality of first-level planning, thus providing more guiding and consistent high-level constraints for subsequent storyboard generation, making the generated storyboard sequence more coherent and expressive in terms of narrative logic and emotional expression.

[0092] In some of the above implementations, key events are identified from event sequences to generate narrative themes and emotional arcs. However, in the process of identifying key events, judging solely based on the surface characteristics of events may not accurately capture the deep influence and importance of events within the entire narrative structure, thereby affecting the quality of subsequent narrative themes and emotional arcs, and potentially causing the generated storyboard sequence to fail to accurately convey the core of the narrative.

[0093] In response, this application further proposes a method for identifying key events from an event sequence, which includes: determining the event influence of each event based on the causal influence range of each event in the event sequence; identifying events whose event influence exceeds a preset influence threshold as candidate key events; and identifying candidate key events as key events.

[0094] For example, when determining the impact of each event based on its causal scope within a sequence of events, this step aims to quantify the importance of the events within the narrative. Causal chain analysis can be performed on the event sequence to identify the antecedent-effect relationships between events. For instance, natural language processing techniques can be used to analyze verbs, nouns, and conjunctions in event descriptions to construct a causal graph between events. The causal scope of an event can be understood as the set of other events directly or indirectly caused by that event, or the set of other events affected by that event. Event impact can then be calculated based on this causal scope; for example, the more subsequent events an event influences, or the more central its position in the causal chain, the greater its impact. This impact can be measured based on the breadth and depth of the event's spread, or its centrality within the narrative structure.

[0095] Events whose influence exceeds a preset influence threshold are identified as candidate key events. After calculating the influence of each event, a preset influence threshold needs to be set. This threshold is a configurable value used to distinguish between ordinary events and events with potential keyness. When the calculated influence value of an event is higher than or equal to the preset influence threshold, the event is considered to have sufficient narrative importance and is thus initially screened as a candidate key event. This threshold can be dynamically adjusted according to the type and length of the narrative text, or the desired number of key events, to adapt to different narrative styles and needs.

[0096] The candidate key event is then identified as the key event. After filtering by influence thresholds, all events identified as candidate key events are confirmed as the key events. This step completes the identification of key events, ensuring that the identified key events are those that have been quantitatively assessed and have narrative impact.

[0097] The aforementioned technical solution overcomes the potential biases of traditional methods in identifying key events, avoiding reliance on superficial features or subjective judgments. By introducing the causal impact of events and quantifying their influence, core events with profound implications for the entire narrative can be identified more objectively and accurately. These precisely identified key events serve as a solid foundation for generating the narrative theme, ensuring that the generated theme truly reflects the story's core themes and deeper meanings. Simultaneously, these key events help to more accurately construct emotional arcs, closely integrating emotional changes with narrative development. This key event identification mechanism based on event influence enhances the accuracy and depth of narrative planning during storyboard generation, resulting in more expressive storyboard sequences that better align with the spirit of the original work.

[0098] In some embodiments described above in this application, a method for identifying key events from an event sequence is proposed, which requires determining the event impact of each event based on the causal influence range of each event in the event sequence. However, in practice, how to accurately and quantitatively assess the causal influence range of an event and transform it into a guiding event impact to ensure the accuracy of key event identification is a problem that needs further refinement and resolution.

[0099] In response, this application further proposes a method for determining the event influence of each event based on the causal influence range of each event in the event sequence. Specifically, this method includes: determining the set of directly affected events and the set of affected events for each event based on the causal influence range; determining the causal propagation depth of each event based on the number of events in the set of directly affected events and the set of affected events; and determining the event influence of each event based on the causal propagation depth.

[0100] For example, when determining the set of directly influencing events and the set of events affected by each event, the system analyzes the causal relationships of each event in the event sequence within the overall narrative structure. The set of directly influencing events refers to the set of all subsequent events directly caused or triggered by the current event, while the set of events affected refers to the set of all preceding events directly caused or triggered by the current event. This process can be achieved by constructing a causal relationship graph of narrative events, where each event is a node and causal relationships are directed edges. By traversing this graph, causal nodes directly connected to the current event can be identified, thereby precisely defining the scope of its direct causal effect. For instance, natural language processing techniques can be used to extract causal connecting words or sentence structures between events from the narrative text, thereby establishing causal chains between events.

[0101] When determining the causal propagation depth of each event, the system counts the number of events in both the set of events it directly affects and the set of events it affects. Causal propagation depth is a quantitative indicator that measures the importance and centrality of an event in the narrative causal chain. The causal propagation depth of an event can be defined as the sum of the number of events in the set of events it directly affects and the number of events in the set of events it affects, or, depending on the specific needs, different weights can be assigned to causal relationships in different directions. For example, if an event directly leads to three subsequent events and is directly triggered by two preceding events, its causal propagation depth can be calculated as five. This quantification method intuitively reflects the connection density of events in the causal network.

[0102] The system determines the impact of each event based on the causal propagation depth. Event impact is an assessment of an event's importance within the narrative. Generally, events with greater causal propagation depth have greater impact on the narrative. A mapping relationship can be established between causal propagation depth and event impact, for example, through a pre-defined function or rule table. For instance, causal propagation depth can be divided into different levels, each corresponding to an event impact value, or linear or non-linear functions can be used to directly convert depth values ​​into impact scores. This method ensures that the assessment of event impact is based on objective causal connection data, rather than subjective judgment.

[0103] The aforementioned technical solution provides a more precise and quantitative method for assessing the impact of narrative events. By clearly defining the set of directly influencing events and the set of affected events, and calculating the causal propagation depth based on their quantity, it effectively avoids vague judgments about the scope of causal influence, thus making the determination of event impact more objective and reliable. This makes the process of identifying key events from an event sequence more accurate, thereby improving the quality of narrative theme generation. For example, when dealing with narrative texts with complex plots and multiple causal chains, this method can effectively distinguish between those events that are merely links in a causal chain and those that are core hubs that play a decisive role in the overall story's direction. This provides a more solid foundation for the subsequent emotional arc and narrative theme generation, enhancing the narrative coherence and expressiveness of the storyboard sequence.

[0104] In some of the embodiments described above in this application, a first-level planning of the narrative element set is proposed to generate an emotional arc and narrative theme. However, when generating the narrative theme directly based solely on key events, it may be difficult to fully capture the deeper meaning and emotional inclination of the narrative text, resulting in a lack of sufficient hierarchy and emotional depth in the generated narrative theme, which in turn affects the accurate expression of the narrative intention in subsequent storyboard generation.

[0105] In response, this application further proposes the following steps for generating the narrative theme based on the key event: determining the core narrative theme of the event sequence based on the key event; determining the emotional tone of the narrative theme based on the core narrative theme and the event type distribution of the event sequence; and merging the core narrative theme and the emotional tone into the narrative theme.

[0106] For example, when determining the core narrative theme of a sequence of events, the core narrative theme is the story's most fundamental and deepest meaning or idea, which is usually embodied through key events. This step aims to extract the core ideas, conflicts, or values ​​that the story expresses from the turning points, climaxes, or decisive events. This can be achieved through Natural Language Processing (NLP) techniques, which perform theme modeling on the textual descriptions of key events to identify high-frequency and representative words or phrases as candidates for the core theme. Alternatively, pre-trained language models can be used to perform semantic analysis on key events, extracting their potential theme vectors, and then using clustering or similarity calculations to determine the core theme that can represent the entire sequence of events. For example, if the key events involve "heroic sacrifice to save the world," the core themes might be "sacrifice," "redemption," and "courage."

[0107] When determining the emotional tone of a narrative theme, the emotional tone refers to the overall emotional tendency inherent in the narrative theme, such as tragedy, comedy, inspiration, or suspense. It is influenced not only by the core theme but also closely related to the distribution of various events (such as positive, negative, and neutral events) in the event sequence. This step aims to imbue the core theme with appropriate emotional color, making it more expressive. This can be achieved by performing sentiment analysis on the core narrative theme to determine its inherent emotional tendency. Simultaneously, each event in the event sequence is sentiment-labeled, and the proportions of positive, negative, and neutral events and their distribution over time are statistically analyzed. Combining the emotional tendency of the core theme and the distribution of event types, a rule engine or machine learning model (such as logistic regression, support vector machine (SVM), random forest, etc.) is used to comprehensively determine the overall emotional tone of the narrative theme. For example, if the core theme is "resistance," and the event sequence has a high proportion of negative events, ending in tragedy, the emotional tone might be "tragic"; if the core theme is "growth," and positive events gradually increase, the emotional tone might be "inspirational."

[0108] The core theme and emotional tone of the narrative are integrated to form the narrative theme. Integration organically combines the extracted core ideas and emotional nuances to create a complete, accurate, and expressive narrative theme. This integrated theme will serve as an important guide for subsequent storyboard generation, ensuring that the storyboards accurately convey the deeper meaning and emotional atmosphere of the story. Integration can be achieved in various ways, such as generating a phrase or sentence containing the core theme and emotional tone, like "a tragic epic about courage and sacrifice," or constructing a structured data object containing "theme" and "emotion" fields. Alternatively, template filling can be used to embed the core theme and emotional tone into a preset narrative theme template to generate a narrative theme in natural language.

[0109] Through the aforementioned technical solution, the generation of the narrative theme no longer relies solely on the direct extraction of key events, but is further refined into the determination of the core narrative theme and the assignment of emotional tone. Extracting the core narrative theme from key events ensures an accurate grasp of the story's deeper meaning. Combining this core theme with the event type distribution of the event sequence allows for a more comprehensive and refined capture of the overall emotional tendency inherent in the narrative text, thereby determining an appropriate emotional tone. The organic integration of the core theme and emotional tone forms a narrative theme that is both profound and emotionally resonant. This step-by-step, refined approach enables the generated narrative theme to more accurately and richly reflect the connotation of the original narrative text, providing more guiding high-level constraints for subsequent scene division and shot breakdown. This effectively improves the accuracy and expressiveness of the storyboard sequence in conveying the story theme and emotions, avoiding the problem of discrepancies between the storyboard content and the original meaning due to misunderstandings of the theme.

[0110] In some of the embodiments described above in this application, an emotional arc is generated based on the distribution of event types in an event sequence. However, in the actual process of generating an emotional arc, how to effectively transform the discrete event type distribution into a continuous arc with a clear narrative structure and emotional direction, so as to ensure that the generated emotional arc can accurately reflect the overall emotional change trend of the story and provide clear emotional guidance for subsequent storyboard generation, is a problem that needs to be solved.

[0111] In response, this application further proposes a step for generating the emotional arc based on the event type distribution of each event in the event sequence, including: dividing the event sequence into four emotional stages (introduction, development, transition, and conclusion) based on the event type distribution; determining an emotional target value for each of the four emotional stages; and arranging the emotional target values ​​of each emotional stage in chronological order to obtain the emotional arc.

[0112] For example, when dividing an event sequence into four emotional stages—introduction, development, climax, and conclusion—based on the event type distribution, this step aims to logically segment the event sequence of the narrative text according to the classic narrative structure, reflecting the emotional fluctuations and development of the story. "Introduction, development, climax, and conclusion," as a common narrative structure, effectively organizes story content, making its emotional expression more layered and predictable. The event type distribution refers to the frequency and pattern of events of different natures (e.g., positive, negative, neutral, conflict, resolution, etc.) occurring in the event sequence. By analyzing these event type distributions, the system can identify key turning points or emotional climaxes in the narrative, thus accurately mapping the entire event sequence to the four stages of "introduction," "development," "climax," and "conclusion." For example, machine learning models (e.g., Hidden Markov Models (HMM), Conditional Random Fields (CRF), Long Short-Term Memory Networks (LSTM)) can be trained on a large number of narrative texts to automatically identify and divide these stages based on the event type distribution; alternatively, a series of rules can be set, such as when the number of positive events increases or conflict events reach a peak, as the basis for stage transitions.

[0113] When determining the emotional target value for each of the four emotional stages (introduction, development, transition, and conclusion), this step aims to assign a quantifiable emotional index to each segmented emotional stage to clarify its emotional tone and intensity. The emotional target value can be a numerical emotional score (e.g., from -1 to 1 representing negative to positive) or a discrete emotional label (e.g., joy, sadness, tension, calm). The value can be determined based on statistical results of the event type distribution within that stage. For example, if the "transition" stage contains numerous conflicts or tragic events, its emotional target value might be set as a strong negative emotion; while the "conclusion" stage, containing resolution or reunion events, might be set as a positive or calm emotion. Furthermore, pre-defined narrative templates or emotional curve models can be used to set typical emotional target values ​​for different stages, with fine-tuning based on the actual event content. Specifically, the emotional target value for each emotional stage is determined according to the following rules: each event in the event sequence is labeled with emotional polarity, with positive events receiving a positive emotional polarity score, negative events receiving a negative emotional polarity score, and neutral events receiving a zero emotional polarity score; this emotional polarity score can be determined by matching a pre-set emotional dictionary (e.g., CNKI emotional dictionary, Dalian University of Technology emotional vocabulary ontology), or by performing emotional analysis on the event description using an emotional analysis model (e.g., an emotional classification model based on Long Short-Term Memory Network (LSTM) or an emotional analysis model based on Transformer); the arithmetic mean or weighted average of the emotional polarity scores of all events within this emotional stage is calculated as the emotional target value for this emotional stage, wherein the weights used in the weighted average are determined based on the event's influence. For example, the "Start" phase includes one neutral event (emotional polarity score of 0) and two slightly positive events (emotional polarity scores of 0.3 each), with an average of 0.2. Therefore, the emotional target value for the "Start" phase is set at 0.2. The "Turn" phase includes two negative events (emotional polarity scores of -0.3 each), with an average of -0.3. Therefore, the emotional target value for the "Turn" phase is set at -0.3.

[0114] When arranging the emotional target values ​​of each emotional stage in chronological order to obtain the emotional arc, this step involves organizing and connecting the previously determined emotional target values ​​of each emotional stage according to their chronological order in the event sequence, thereby forming a complete and continuous trajectory of emotional change. This emotional arc visually demonstrates the emotional evolution of the story from beginning to end, for example, from a stable "beginning" to a gradually rising "development," then to a climaxing "turn," and finally to a stable or positive "conclusion." This chronological arrangement ensures the synchronization of the emotional arc with the narrative progress, providing clear emotional guidance on a timeline for subsequent storyboard generation.

[0115] By combining the abstract distribution of event types with traditional narrative structure theory, the event sequence is divided into four emotional stages—introduction, development, climax, and conclusion—each with a clear narrative function and emotional trajectory. Specific emotional target values ​​are assigned to each stage and arranged chronologically, thus generating a clearly structured and logically coherent emotional arc. This not only solves the problem of transforming discrete event information into a continuous emotional trajectory but also provides precise and expressive emotional constraints for subsequent scene planning and shot decomposition, ensuring the accuracy and artistry of the storyboard sequence in emotional expression. This allows the generated storyboards to better convey the deeper emotional intent of the narrative text.

[0116] In some of the embodiments described above in this application, an event sequence is divided into four emotional stages—introduction, development, transition, and conclusion—based on the distribution of event types. However, in practice, relying solely on the distribution of event types for a rough division may result in blurred boundaries between emotional stages, making it difficult to accurately capture subtle changes in the emotional tone and key turning points in the narrative text, thereby affecting the accuracy of the generated emotional arc and the coherence of the narrative logic.

[0117] In response, this application further proposes a step to divide the event sequence into four emotional stages based on the event type distribution: determining the emotional turning point in the event sequence based on the event type distribution; dividing the event sequence into consecutive event segments based on the emotional turning point; and mapping the event segment to the four emotional stages based on the event type of the event in each event segment and the position of each event segment in the event sequence.

[0118] An emotional turning point refers to a key event or set of events in a narrative text that alters the emotional tone, narrative rhythm, or character state. Identifying emotional turning points is fundamental to accurately dividing the narrative into stages, ensuring that subsequent emotional phases accurately reflect the story's emotional trajectory. For example, it can be identified based on the distribution of event types within a sequence of events, combined with pre-defined rules of emotional change or machine learning models (such as sentiment classification models based on Long Short-Term Memory (LSTM) networks or Support Vector Machines (SVMs). For instance, an emotional turning point can be identified when a series of negative events follows a series of positive events, or when the causal influence of an event causes a reversal in the emotional polarity of subsequent events.

[0119] An event segment refers to a narrative unit composed of consecutive events, with its boundaries defined by emotional turning points. By logically dividing the entire event sequence based on these identified emotional turning points, subsequences that are emotionally consistent or share a specific emotional trajectory can be obtained. For example, if the event sequence is E1-E2-E3-E4-E5-E6, and E3 and E5 are identified as emotional turning points, then the event sequence will be divided into three consecutive event segments: [E1, E2], [E3, E4], and [E5, E6]. This segmentation method provides a structured basis for subsequently mapping event segments to the introduction, development, transition, and conclusion phases.

[0120] Mapping refers to associating segmented event paragraphs with four predefined emotional stages: "introduction," "development," "transition," and "conclusion." This step aims to assign a clear narrative function and emotional positioning to each event paragraph, thereby constructing a complete emotional arc. In practice, based on the distribution of event types within each paragraph (e.g., positive events, negative events, conflict events, resolution events, etc.) and the paragraph's relative position in the entire event sequence (e.g., initial, middle, and later stages), it can be matched to one of the "introduction," "development," "transition," or "conclusion" stages using pre-defined mapping rules or a trained classification model. For example, an initial paragraph that mainly lays the groundwork and introduces the event might be mapped to "introduction," a middle paragraph where conflict intensifies and emotional fluctuations are significant might be mapped to "transition," and a later paragraph where conflict is resolved and emotions stabilize might be mapped to "conclusion."

[0121] The aforementioned technical solution accurately identifies emotional turning points in narrative texts, avoiding the blurred boundaries of emotional stages caused by rough division based solely on event type distribution. By dividing the event sequence into continuous event segments using emotional turning points as boundaries, it ensures that each segment has relative emotional consistency or a clear direction. Combining the event types within each segment with their position in the overall event sequence, these are mapped to four emotional stages: introduction, development, transition, and conclusion. This makes the generation of emotional arcs more consistent with narrative logic and the laws of emotional development, improving the accuracy and coherence of emotional expression during storyboard generation. Consequently, it provides more refined and reliable emotional constraints for subsequent scene planning and shot breakdown.

[0122] While some of the aforementioned implementations propose generating emotional arcs and narrative themes as the basis for subsequent planning, these high-level narrative elements, when directly applied to specific scene planning and shot decomposition, may be difficult to precisely guide subsequent operations due to their abstract nature, leading to deviations between the planning results and the intended narrative intent. For example, the narrative theme might be a complex sentence or paragraph, while the emotional arc might consist of a series of discrete emotional target values; these forms are not conducive to quantitative analysis and matching by the system.

[0123] To address this, this application further proposes transforming the narrative theme into a set of thematic keywords and the emotional arc into emotional range constraints, based on pre-defined refinement rules. For example, see... Figure 4 The method includes: 401. Based on the preset refinement rules, extract keywords from the narrative theme to obtain a set of theme keywords.

[0124] The narrative theme typically exists in the form of natural language text, and its content may be broad or abstract. To enable its effective use by subsequent automated processing modules, it needs to be transformed into a more operational form. Keyword extraction aims to identify words or phrases from the narrative theme that represent its core content, theme, or intention. For example, statistical methods such as TF-IDF (Term Frequency-Inverse Document Frequency) or graph ranking algorithms such as TextRank can be used to automatically identify highly important words from the narrative theme text. Alternatively, pre-trained language models can be used for semantic analysis to extract conceptual words related to the narrative theme. These extracted keywords collectively constitute a set of topic keywords, providing concrete and quantifiable evidence for subsequent semantic association and constraints.

[0125] 402. Convert the emotional target values ​​of each stage in the emotional arc into corresponding numerical ranges.

[0126] The emotional arc typically describes the dynamic changes in emotional intensity during the narrative process, and its emotional target value may be a precise numerical point. However, in the actual storyboard generation process, overly precise emotional target values ​​may limit creative flexibility. Therefore, by expanding each emotional target value into a numerical range, a permissible fluctuation range can be provided for subsequent scene emotional allocation. For example, based on a preset tolerance or emotional intensity level, an upper and lower limit can be set for each emotional target value, centered on or bounded by it, thus forming a range. Specifically, the rules for determining the numerical range are as follows: Using the emotional target value as the center, determine the half-width based on the emotional intensity level of that emotional stage; subtract and add this half-width to the emotional target value respectively to obtain the lower and upper limits of the numerical range; where the higher the emotional intensity level, the larger the half-width. For example, the emotional target value for the "beginning" stage is 0.2, corresponding to a "weak" emotional intensity level, with a half-width of 0.1. Therefore, the numerical range is [0.2-0.1, 0.2+0.1], or [0.1, 0.3]. The emotional target value for the "transition" stage is -0.3, corresponding to a "strong" emotional intensity level, with a half-width of 0.2. Therefore, the numerical range is [-0.3-0.2, -0.3+0.2], or [-0.5, -0.1]. This range maintains the overall direction of the emotional arc while allowing for some fine-tuning of the scene's emotional tone to adapt to specific narrative needs.

[0127] 403. Combine the numerical intervals in chronological order to obtain the emotional interval constraints.

[0128] The emotional arc itself has a clear temporal dimension, and the emotional target values ​​and their corresponding numerical ranges at each stage should also follow this chronological order. By integrating these chronologically arranged numerical ranges, a continuous sequence of emotional constraints covering the entire narrative timeline can be formed. This sequence clearly defines the intensity range that scene emotions should satisfy at different stages of the narrative. This combination method ensures the integrity and coherence of emotional constraints, providing global guidance for subsequent scene emotion allocation, and enabling the storyboard generation process to better follow the preset emotional rhythm.

[0129] Through the aforementioned technical solution, the abstract narrative theme is transformed into a specific set of thematic keywords. This allows the system to more accurately understand and grasp the core intent of the narrative, providing a clear quantitative basis for the narrative function annotation of subsequent scenes and the semantic association of thematic keywords, thus ensuring the consistency between the storyboard content and the narrative theme. Simultaneously, the discrete emotional target values ​​in the emotional arc are transformed into continuous numerical ranges and combined chronologically into emotional range constraints. This provides flexible and bounded guidance for the emotional allocation of scenes, avoiding the rigidity that a single target value might bring. It allows for appropriate adjustments to the scene's emotion while maintaining the overall emotional direction, thereby enhancing the richness of emotional expression and narrative coherence in the generated storyboard. This refinement and transformation mechanism effectively bridges the gap between high-level narrative concepts and low-level storyboard details, making the storyboard generation process more controllable and artistic.

[0130] In some of the embodiments described above in this application, in order to transform the narrative theme into a set of thematic keywords, keywords are typically extracted from the narrative theme according to preset refinement rules. However, in practice, simple keyword extraction may be insufficient to accurately capture the deep core intent contained in the narrative theme, resulting in an inaccurate or incomplete set of thematic keywords, thereby affecting the accuracy and artistic expression of subsequent scene planning and shot breakdown.

[0131] In response, this application further proposes to semantically decompose the narrative theme according to preset refinement rules to obtain a sequence of semantic units; to select semantic units that represent the core intention of the narrative from the sequence of semantic units; and to determine the selected semantic units as the thematic keywords in the set of thematic keywords.

[0132] For example, in the process of transforming the narrative theme into a set of thematic keywords, the narrative theme is semantically decomposed according to pre-defined refinement rules to obtain a sequence of semantic units. Semantic decomposition aims to break down complex narrative theme statements into smaller, more independent semantic units. These units can be words, phrases, or concepts, which together constitute the deep semantic structure of the narrative theme. For instance, techniques such as dependency parsing, semantic role labeling, or abstract meaning representation from natural language processing can be used to identify the subject-verb-object structure, modification relationships, and core concepts in the narrative theme, thereby decomposing it into a series of clear semantic units. Pre-defined refinement rules play a guiding role in this process, ensuring that the decomposition process effectively reveals the key information of the narrative theme and provides a foundation for subsequent filtering.

[0133] The process involves filtering semantic units from this sequence that represent the core narrative intent. The core narrative intent refers to the fundamental theme, emotional inclination, or core conflict that the narrative aims to express. This filtering can be achieved in various ways. For example, semantic units can be ranked according to their importance score within the narrative's main theme. This score can be calculated based on their frequency of occurrence in the text, positional weight, or semantic similarity to a pre-defined narrative theme database. Alternatively, pre-trained semantic models can be used to classify or cluster semantic units, identifying those related to the core narrative theme. This approach effectively filters out secondary or decorative information from the narrative's main theme, focusing on the semantic content that represents its essence.

[0134] The selected semantic units are designated as thematic keywords in the thematic keyword set. These selected semantic units, because they concisely and accurately express the core narrative intent, can be directly used as components of the thematic keyword set. In some cases, to maintain the conciseness and standardization of keywords, these semantic units can be further standardized, such as simplifying complex phrases into core words or mapping them to a predefined keyword system. Through these steps, it is ensured that the generated thematic keyword set is not only concise in number, but that each keyword carries an important core narrative intent.

[0135] By employing the aforementioned technical solutions, semantic decomposition of the narrative theme can reveal its inherent semantic structure and constituent elements more deeply, avoiding the loss or bias of semantic information that may result from simple keyword extraction. Further screening identifies semantic units representing the core narrative intent, ensuring that the extracted thematic keywords accurately reflect the core ideas and deeper meanings of the narrative, rather than merely remaining at the surface level. These precisely selected semantic units are defined as a set of thematic keywords, making the set not only comprehensive but also concise, more effectively guiding the scene narrative function annotation in the second-level planning, ensuring that the narrative function of each scene remains consistent with the overall narrative theme. Furthermore, these precise thematic keywords, when subsequently mapped to narrative intent parameters, can more accurately guide shot decomposition, thereby generating a more expressive storyboard sequence that better aligns with the creator's intentions, enhancing the artistry and narrative coherence of the generated storyboard.

[0136] In some of the aforementioned implementations, the emotional target values ​​at each stage of the emotional arc are proposed to be transformed into corresponding numerical ranges to form emotional range constraints. However, when directly mapping abstract emotional target values ​​to numerical ranges, challenges may arise in accurately capturing and quantifying subtle changes in emotion, and ensuring that the generated numerical ranges can effectively guide subsequent scene emotional planning. If the transformation process lacks meticulous consideration, the expression of emotional range constraints may be imprecise, thus affecting the accurate grasp of emotional changes during storyboard generation.

[0137] In response, this application further proposes a method for converting the emotional target values ​​of each stage in the aforementioned emotional arc into corresponding numerical ranges, which includes: determining the emotional intensity level of the emotional target value of each stage in the emotional arc; determining the upper limit and lower limit of the numerical range corresponding to each stage based on the emotional intensity level; and combining the upper limit and the lower limit to form the numerical range.

[0138] For example, when determining the emotional intensity level of the emotional target value at each stage of the emotional arc, the aim is to quantitatively evaluate the emotional target value represented by each stage to clarify the intensity of its emotion. The determination of emotional intensity levels can be achieved in several ways. For instance, an emotional intensity level scale can be pre-defined, such as multiple levels like "very weak," "weak," "moderate," "strong," and "very strong." Then, natural language processing (NLP) techniques can be used to perform sentiment analysis on the text descriptions corresponding to the emotional target values, mapping their sentiment scores to the pre-defined levels. Alternatively, machine learning models (such as Support Vector Regression (SVR), Random Forest Regression, and Neural Network Regression models) can be used to directly predict the emotional intensity level of each emotional target value by training on a large dataset labeled with emotional intensity. Furthermore, expert knowledge can be incorporated, with domain experts manually evaluating and labeling the emotional target values ​​to determine their emotional intensity levels.

[0139] When determining the upper and lower limits of the numerical range corresponding to each stage based on the emotional intensity level, this step uses the level as a basis to set the specific range of the corresponding numerical range, i.e., the upper and lower limits. For example, a mapping rule can be preset to associate different emotional intensity levels with different numerical ranges. For a "weak" emotional intensity level, a narrower numerical range close to the neutral value can be set (e.g., [0.1, 0.3], i.e., an interval centered at 0.2 with a half-width of 0.1); for a "strong" emotional intensity level, a wider numerical range deviating from the neutral value can be set (e.g., [-0.5, -0.1], i.e., an interval centered at -0.3 with a half-width of 0.2). This mapping relationship can be adjusted and optimized according to specific narrative needs and emotional expression habits. In addition, a dynamic adjustment strategy can also be adopted, adaptively calculating appropriate upper and lower limits based on the emotional intensity level of the current stage and its connection with adjacent stages to ensure the smoothness and rationality of emotional changes.

[0140] The upper and lower limits are combined to form the numerical range. This step integrates the previously determined upper and lower limits to form a complete numerical range. For example, if the upper limit is 0.9 and the lower limit is 0.7, the range [0.7, 0.9] is formed. This numerical range represents the range of emotional values ​​allowed for this emotional stage, providing a clear quantitative constraint for subsequent scene emotion allocation.

[0141] Through the aforementioned technical solution, the conversion of emotional target values ​​at each stage of the emotional arc into numerical ranges is no longer a simple direct mapping, but rather a refined assessment of the emotional intensity levels of the emotional target values. This assessment makes the determination of the upper and lower limits of the subsequent numerical ranges more based on evidence and more flexible. For example, by introducing emotional intensity levels, the width and position of the numerical range can be dynamically adjusted according to the intensity of the emotion, thereby more accurately capturing and quantifying the complex emotional changes contained in the narrative text. For instance, for stages with high emotional intensity, a broader or more extreme numerical range can be set to allow for a larger space for emotional expression; while for stages with low emotional intensity, a narrower or more neutral range can be set to ensure the introspection of emotional expression. This refined conversion method allows the generated emotional range constraints to more accurately reflect the emotional intent of the narrative text, providing more reliable and detailed guidance for the allocation of emotional change targets in subsequent scenes, thereby improving the accuracy and artistic appeal of the expression of narrative emotions during the storyboard generation process.

[0142] In some of the embodiments described above in this application, although the emotional target values ​​of each stage in the emotional arc can be converted into corresponding numerical ranges, if these numerical ranges lack a clear temporal order and organizational structure, they are difficult to effectively guide subsequent scene planning. Simply listing these numerical ranges cannot accurately reflect the dynamic evolution of narrative emotions, which may lead to insufficient coherence in emotional guidance during storyboard generation or inconsistency with the narrative theme.

[0143] In response, this application further proposes a method for combining the numerical intervals in chronological order to obtain the emotional interval constraint. The method includes: determining the time position of each numerical interval in the emotional arc; arranging the numerical intervals in chronological order according to the time position; and organizing the numerical intervals in the order of arrangement to obtain the emotional interval constraint.

[0144] For example, when determining the temporal position of each numerical interval within the emotional arc, the aim is to clarify the specific location of each interval within the entire narrative timeline. The emotional arc itself is arranged chronologically, reflecting the emotional trends of the narrative. Therefore, when the emotional arc is divided into different stages, and the emotional target value of each stage is converted into numerical intervals, it is necessary to trace the initial temporal context of these numerical intervals within the emotional arc. This can be achieved by recording the start and end times of the emotional stage corresponding to each numerical interval within the original emotional arc, or their relative order and duration throughout the narrative process. For instance, a timestamp or a time period identifier can be associated with each numerical interval to ensure that subsequent combinations and arrangements accurately reflect the narrative's temporal logic and the order of emotional evolution, laying the foundation for constructing coherent emotional interval constraints.

[0145] Based on the time position, the numerical intervals are arranged in chronological order. After determining the time position of each numerical interval, this step reorganizes these discrete numerical intervals according to their chronological order on the narrative timeline. This typically involves a sorting algorithm; for example, if the time position is a timestamp, it is sorted in ascending order of timestamp; if the time position is a stage number, it is sorted in ascending order of the stage number. The result of the arrangement is an ordered sequence of numerical intervals that faithfully reflects the dynamic development of narrative emotion, thus establishing a logical order between emotional intervals. This allows them to function as a whole, guiding the subsequent emotional allocation in scenes and ensuring that emotional changes align with the narrative's temporal progression.

[0146] Organizing the numerical intervals according to this arrangement order yields the emotional interval constraint. This step involves structurally encapsulating the numerical intervals already arranged in chronological order to form a unified and operable emotional interval constraint. The organization method can be to store these ordered numerical intervals in a list, array, or define a specific data structure containing these intervals. This data structure not only contains the numerical intervals themselves but also implicitly implies the temporal order relationship between them, thus constituting a complete and dynamic emotional constraint model.

[0147] The aforementioned technical solution ensures that after converting the emotional target values ​​of each stage in the emotional arc into numerical ranges, these ranges can be effectively combined according to their true order on the narrative timeline. By clearly defining the temporal position corresponding to each numerical range and precisely sorting them, the resulting emotional range constraints not only encompass the range of emotional intensity at each stage, but more importantly, they completely preserve the dynamic evolution trajectory of narrative emotions. This allows subsequent scene planning to strictly adhere to the narrative's temporal logic and emotional development patterns when allocating emotional change targets, avoiding problems such as incoherent emotional expression or contradictions with the narrative theme caused by disordered emotional range order, thereby improving the accuracy and effectiveness of emotional guidance during storyboard generation.

[0148] In some of the embodiments described above in this application, although semantic analysis of the narrative text has been proposed to generate emotional arcs and narrative themes, and event sequences have been planned based on these, how to ensure the rationality of scene division and avoid disrupting the causal logic between events when transforming abstract event sequences into specific storyboard scenes, while effectively integrating the narrative themes and emotional constraints into the narrative function of each scene to guide subsequent shot decomposition, remains a problem that needs to be solved.

[0149] In response, this application proposes to use the set of thematic keywords and the emotional range constraint as input constraints to perform a second-level planning on the event sequence in the narrative element set, thereby obtaining the scene sequence and the narrative function labeling of each scene.

[0150] For example, see Figure 5 This includes the following steps.

[0151] 501. Based on the event boundaries and narrative element set of each event in the event sequence, the event sequence is divided into scenes to obtain the scene sequence.

[0152] Here, the event boundary refers to the start and end points of the smallest semantic unit constituting an event, such as changes in time, place, characters, or actions. The narrative element set provides contextual information about the event's occurrence, including character entities, scene entities, event sequences, and character relationships. A key constraint when dividing scenes is ensuring that the scene boundaries do not interrupt the causal chain of events in the event sequence. This means that if there is a direct causal relationship between two or more events—for example, one event is the direct cause or result of another—then, where possible, these causally related events should be included in the same scene, or the scene boundary should be adjusted to the logical breakpoint of the causal chain to maintain the integrity and coherence of the narrative logic and avoid confusion for the audience when understanding the story's development.

[0153] 502. Using the emotional interval constraint as the input constraint, assign an emotional change target to each scene in the scene sequence.

[0154] The emotional range constraint, refined from the overall emotional arc, sets the scope or trend for the emotional trajectory of the narrative at different stages. Assigning emotional change goals to each scene aims to refine this macro-level emotional planning down to the scene level. For example, a scene might be assigned an emotional progression goal from "calm" to "tension," or an emotional shift goal from "sadness" to "hope." This assignment can be based on the scene's position within the overall narrative, the nature of the events it contains, and its contribution to the overall emotional arc. By setting clear emotional goals for each scene, a clear emotional direction can be provided for subsequent shot design and visual content description, ensuring that each scene effectively conveys the intended emotional experience.

[0155] 503. Based on the set of events and thematic keywords contained in each scene, determine the narrative function label for each scene.

[0156] Narrative function annotation is a general description of the role a scene plays in the overall story, such as "introducing conflict," "revealing the truth," or "character growth." Determining this annotation requires in-depth analysis of the specific events occurring within the scene and semantically associating them with a set of thematic keywords (representing the core theme and intention of the narrative). For example, if an event in a scene describes the protagonist facing and overcoming a predicament, and the thematic keyword set contains the word "courage," then the narrative function annotation for that scene might be determined as "demonstrating the protagonist's courage." Crucially, the narrative function annotation for each scene must establish a semantic association with at least one thematic keyword in the thematic keyword set. This ensures that each scene closely revolves around the core theme, preventing the scene content from becoming disconnected from the narrative's main idea, thus making the overall story structure more compact and the theme expression clearer.

[0157] The aforementioned technical solution effectively transforms abstract event sequences into clearly structured and logically coherent scene sequences. By strictly adhering to the causal chain integrity of events during scene division, narrative logical breaks are avoided, ensuring the story's fluency and comprehensibility. Simultaneously, refining emotional range constraints to the emotional change targets of each scene allows for precise representation and control of the overall emotional arc at the scene level, thereby enhancing the narrative's emotional impact. Establishing semantic connections between the narrative function annotations of each scene and a set of thematic keywords ensures that each scene closely serves the core narrative theme, making the story's central message more focused and powerful. Overall, this hierarchical planning method provides a solid foundation for subsequent shot decomposition, resulting in a storyboard sequence that is not only visually appealing but also unified and optimized in terms of narrative logic, emotional expression, and thematic depth.

[0158] In some of the embodiments described above in this application, when dividing an event sequence into scenes, it is necessary to ensure that the boundaries of the scene division do not truncate the causal chain of events in the event sequence. However, in its implementation, simply dividing based on event boundaries may result in key causal relationships being inappropriately segmented into different scenes, thereby affecting the coherence of the narrative logic and the audience's understanding of the story.

[0159] To address this, this application further proposes a method for dividing the event sequence into scenes based on the event boundaries and narrative element sets of each event in the event sequence, thereby obtaining a scene sequence. The boundaries of this scene division do not truncate the causal chain of events within the event sequence. Specifically, this includes: Based on the connections between adjacent events in the event sequence, the event boundaries of each event are determined. This step aims to accurately identify the start and end points of each independent narrative unit in the event sequence. These connections can be analyzed based on various dimensions, such as semantic associations, temporal order, spatial changes, or interactions between involved characters. For example, natural language processing techniques can be used to identify event verbs, time adverbs, place adverbs, and participating characters to determine the closeness and logical continuity between events. Determining event boundaries provides a refined foundation for subsequent scene segmentation, ensuring that each event is considered as a complete unit.

[0160] Based on the event boundaries and the set of narrative elements, the event sequence is segmented to obtain candidate scenes. After determining the event boundaries of each event, the system uses these boundaries as initial segmentation points to divide the entire event sequence into a series of preliminary scene fragments, i.e., candidate scenes. During this process, the set of narrative elements, including the event sequence itself and information such as character relationships, can serve as auxiliary criteria for judgment. For example, when there are significant changes in character relationships or the introduction of a new key character, even if the event boundaries are not obvious, they can still serve as a reference for segmenting candidate scenes. This step generates a preliminary scene structure, laying the foundation for subsequent fine-tuning.

[0161] A causal chain integrity check is performed on the boundaries of the candidate scenarios. This is a crucial step in ensuring the coherence of the narrative logic. This check aims to detect whether causal relationships are improperly truncated by scenario boundaries within the initially segmented candidate scenarios. For example, the system analyzes event pairs that cross candidate scenario boundaries to determine if one event is the cause of another, even though these two events are separated by different scenario boundaries. For instance, if event A (cause) occurs at the end of scenario 1, and event B (effect) occurs at the beginning of scenario 2, and there is a strong causal relationship between them, then the scenario boundary may have truncated the causal chain.

[0162] The candidate scene boundaries for truncated causal chains are adjusted to the endpoints of those chains to obtain the scene sequence. Once a truncated causal chain is identified through integrity checks, the system corrects the corresponding candidate scene boundaries. The principle of adjustment is to ensure that complete causal relationships are contained within the same scene, or to switch scenes at the logical endpoint of the causal chain. For example, if event A causes event B, and they are separated by a scene boundary, that boundary will be moved so that events A and B are both in the same scene, thus maintaining the integrity of the narrative logic. Through this adjustment, the resulting scene sequence effectively avoids breaks in causal chains, ensuring the story's fluency and comprehensibility.

[0163] The aforementioned technical solution addresses the issue of truncated causal chains during scene segmentation. By accurately identifying the connections between adjacent events in an event sequence, event boundaries are determined, providing a refined foundation for subsequent scene segmentation. After generating candidate scenes, a causal chain integrity verification mechanism is introduced to proactively detect and identify scene boundaries that may disrupt narrative logic. Adjusting the boundaries of candidate scenes with truncated causal chains ensures the integrity of causal relationships within and between scenes, thereby guaranteeing the coherence of narrative logic and the smoothness of the story. This not only improves the quality of the scene sequence but also provides a more solid foundation for subsequent emotional change target allocation and narrative function annotation, making the generated storyboard sequence more narratively plausible and engaging.

[0164] When dividing an event sequence into scenes to obtain candidate scenes, segmenting solely based on event boundaries may inappropriately truncate key causal relationships, causing the starting and ending events of a causal chain to belong to different scenes. This method of scene segmentation disrupts the coherence of narrative logic, affects the emotional allocation of subsequent scenes and the accuracy of shot breakdown, thereby reducing the narrative quality of the storyboard sequence.

[0165] In response, this application further proposes a method for performing causal chain integrity verification on the boundaries of the aforementioned candidate scenarios, specifically including: determining whether there is a cross-boundary causal relationship pair at the boundary of each candidate scenario; judging whether the causal chain is truncated by the boundary of the candidate scenario based on the candidate scenario where the cause event and the result event in the causal relationship pair are located; and marking the judgment result as the verification status of the candidate scenario.

[0166] The step of determining whether a cross-boundary causal relationship exists at the boundary of each candidate scene refers to the system checking the boundaries between each candidate scene after the initial scene segmentation of the event sequence. This step aims to identify whether there is a situation where an event (causal event) is located in a candidate scene before the boundary, while another event (result event) that has a direct causal relationship with that event is located in a candidate scene after the boundary. This can be achieved by traversing the identified event causal relationship graph and comparing the candidate scenes to which the two events in the causal relationship pair belong. For example, if event A causes event B, and event A is in scene S1, event B is in scene S2, and there is a boundary between S1 and S2, then a cross-boundary causal relationship pair is considered to exist at that boundary.

[0167] Based on the candidate scenarios where the causal event and the result event reside in the causal pair, determining whether the causal chain is truncated by the boundary of the candidate scenario means that after identifying cross-boundary causal pairs, the system further determines whether the causal relationship constitutes an actual truncation of the narrative causal chain. If a direct causal pair is separated by a scenario boundary, i.e., the causal event and the result event belong to different candidate scenarios, then the causal chain is considered to be truncated by that boundary. This determination ensures that only those causal relationships that truly affect the logical coherence of the narrative are identified.

[0168] Marking the judgment result as the verification status of the candidate scene means that after completing the above judgment, the system will record the verification result of each candidate scene boundary. For example, a Boolean flag (such as "causal chain truncation: yes / no") or a status value (such as "normal" or "truncation") can be set for each boundary. These markings provide a clear basis for subsequent scene boundary adjustments, enabling the system to specifically correct those scene boundaries that disrupt the integrity of the causal chain.

[0169] The aforementioned technical solution enables the accurate identification and marking of candidate scene boundaries that inappropriately truncate the narrative causal chain when dividing event sequences into scenes. This effectively avoids narrative logic breaks caused by improper scene division, ensuring narrative coherence within and between each scene. This verification mechanism provides crucial feedback for subsequent scene boundary adjustments, allowing the generated scene sequences to more accurately allocate emotional change targets and label narrative functions while maintaining narrative integrity. This lays a solid foundation for generating high-quality, logically rigorous storyboard sequences.

[0170] In some of the embodiments described above in this application, an emotional range constraint is proposed as input to assign emotional change targets to each scene in a scene sequence. However, in practice, simply assigning emotional change targets independently to each scene based on the emotional range constraint may lead to abrupt or disjointed emotional changes between adjacent scenes, thereby affecting the emotional flow of the overall narrative and the audience's immersion. This allocation method, lacking emotional connection between scenes, may not effectively support the emotional arc required by the narrative theme, making the emotional expression of the scene sequence appear stiff or unnatural.

[0171] In response, this application further proposes a method for assigning emotional change targets to each scene in a scene sequence using emotional interval constraints as input constraints, specifically including: Based on the emotional interval constraint, determine the target range of emotional changes for each scene in the scene sequence; Assign initial emotional change targets to each scenario from the range of emotional change targets; Based on the initial emotional change targets of adjacent scenes in the scene sequence, the initial emotional change targets of each scene are adjusted to obtain the emotional change targets.

[0172] For example, the emotional range constraint is refined from the overall emotional arc of the narrative text (generated by the first-level planning). It sets the allowable range of emotional intensity or tendency for the narrative at different time stages. When assigning emotional change targets to each scene in the scene sequence, the system extracts the corresponding emotional range from the emotional range constraint based on the scene's position on the entire narrative timeline. For instance, if the emotional range constraint defines the emotional range to be between "tension" and "climax" within a certain time period, then the emotional change targets for all scenes within that time period must fall within this preset range. This ensures that the emotional target of each scene is consistent with the emotional direction of the overall narrative, avoiding emotional deviation. The system assigns an initial emotional change target to each scene within the defined emotional change target range. This initial target can be the center value of the range or a value derived from a preliminary assessment based on the scene content (such as the types of events included in the scene, character interactions, etc.). For example, if a scene contains conflict events, its initial emotional target might be set to the negative or tense end of the range; if it contains resolution or reconciliation events, it might be set to the positive or mild end. This initial allocation provides the foundation for subsequent fine-tuning. To ensure a natural and smooth emotional transition in the scene sequence, the system further considers the emotional connection between adjacent scenes. Specifically, it analyzes the relationship between the initial emotional change target of the current scene and the initial emotional change target of the previous or next scene. If it finds excessively large jumps or unreasonable reversals between the emotional targets of adjacent scenes, the system will correct the initial emotional change target of the current scene. This adjustment aims to smooth the emotional curve, making emotional changes conform to the continuity and logic of human perception, for example, by limiting the magnitude of differences between emotional targets of adjacent scenes, or by ensuring the reasonable continuation of emotional trends.

[0173] The aforementioned technical solution addresses the potential abruptness and inconsistency in assigning emotional change targets to scene sequences. By defining the range of emotional change targets for each scene based on emotional interval constraints, it ensures a macro-consistency between the emotional targets of individual scenes and the overall emotional arc of the narrative. After the initial assignment, adjustments are made by considering and adjusting the initial emotional change targets of adjacent scenes, resulting in smoother and more natural emotional transitions between scenes. This adjustment mechanism avoids abrupt emotional rises and falls or illogical shifts, thus guaranteeing the coherence and fluency of the emotional expression in the storyboard sequence. This contributes to generating a storyboard sequence with a more reasonable and impactful emotional rhythm, enhancing the audience's immersion and understanding and resonance with the narrative content.

[0174] In some embodiments described above in this application, an emotional range constraint is used as an input constraint to assign an initial emotional change target to each scene in the scene sequence. However, in practical applications, if the initial emotional change target is assigned independently based solely on the emotional change target range of each scene, it may result in overly drastic or disjointed emotional changes between adjacent scenes, thereby affecting the smoothness of the entire narrative flow and the audience's emotional experience. Such a lack of smooth transitions in emotional changes may make the storyboard sequence appear stiff or unnatural in its emotional expression, making it difficult to effectively guide the audience's emotions.

[0175] In response, this application further proposes a method for adjusting the initial emotional change target of each scene based on the initial emotional change target of adjacent scenes in the scene sequence to obtain the emotional change target. Specifically, the method includes: determining the progressive difference between the initial emotional change targets of adjacent scenes in the scene sequence; determining the scene that needs to be adjusted based on the progressive difference and the magnitude of the range of emotional change targets; and adjusting the initial emotional change target of the scene that needs to be adjusted to the boundary value of the range of emotional change targets to obtain the emotional change target.

[0176] For example, when determining the progressive difference of the initial emotional change target between adjacent scenes in a scene sequence, the initial emotional change targets of any two adjacent scenes in the scene sequence can be compared, and the numerical difference or level difference between them can be calculated. For instance, if the initial emotional change target is represented numerically, the initial emotional change target of the later scene can be directly calculated by subtracting the initial emotional change target of the earlier scene from the initial emotional change target of the later scene, resulting in a directional progressive difference; if it is represented by emotional level, a level distance function can be defined to quantify the difference. This step aims to quantify the intensity or smoothness of emotional changes between adjacent scenes, providing a quantitative basis for subsequent adjustments.

[0177] Based on the magnitude of the progressive difference and the target range of emotional change, scenarios requiring adjustment are identified. This step identifies scenarios where emotional changes do not meet the expected smoothness or coherence requirements by comparing the calculated progressive difference with a preset threshold or the magnitude of the target range of emotional change. For example, a maximum allowable progressive difference threshold can be set; if the progressive difference between adjacent scenarios exceeds this threshold, the emotional change is considered too drastic and requires adjustment. Alternatively, a proportional threshold can be dynamically determined based on the overall magnitude of the target range of emotional change (i.e., the difference between the upper and lower limits). For example, when the progressive difference exceeds a certain percentage of this magnitude, the scenario is marked as requiring adjustment.

[0178] The initial emotional change target for a scene requiring adjustment is adjusted to the boundary value of its emotional change target range, thus obtaining the emotional change target. For scenes determined to require adjustment, their initial emotional change targets are revised. For example, if the emotional progression difference between scenes S_i and S_{i+1} is too large, and the initial emotional change target of S_{i+1} is much higher than that of S_i, the initial emotional change target of S_{i+1} can be adjusted to the lower limit of its emotional change target range, or adjusted to a value closer to S_i, while still ensuring that the adjusted value is within its allowed emotional change target range. Conversely, if the initial emotional change target of S_{i+1} is much lower than that of S_i, it can be adjusted to the upper limit of its emotional change target range. This adjustment strategy aims to make the emotional changes between adjacent scenes smoother, consistent with narrative logic, and avoid extreme emotional jumps.

[0179] The aforementioned technical solution effectively identifies and corrects instances of disjointed or abrupt emotional transitions in scene sequences. By quantifying the difference in emotional progression between adjacent scenes and combining this with the magnitude of the emotional change target range, the scenes requiring adjustment can be precisely determined. The initial emotional change targets of these scenes are corrected to their boundary values, ensuring a more natural and coherent emotional flow throughout the scene sequence and avoiding abrupt emotional jumps. This not only enhances the narrative fluency and audience immersion of the storyboard sequence but also makes the generated storyboard sequence more consistent with human perception and narrative logic in its emotional expression, thereby improving the quality of storyboard generation.

[0180] In some of the embodiments described above in this application, the narrative function annotation of each scene is determined based on the set of events and thematic keywords contained in each scene. However, in practice, ensuring that the determined narrative function annotations not only accurately reflect the scene content but also establish a strong semantic connection with the overall narrative's set of thematic keywords, avoiding arbitrary annotations or deviations from the main theme, is a problem that requires careful handling. Simply annotating based on events and thematic keywords may result in inaccurate narrative function annotations or weak connections with the core theme, thereby affecting the quality of subsequent shot breakdowns and the overall narrative coherence of the storyboard sequence.

[0181] In response, this application further proposes a step-by-step approach to determine the narrative function annotation of each scene based on the events contained in each scene and the set of keywords for that theme, including: Based on the events contained in each scene, determine the candidate set of narrative functions for each scene; Determine the semantic relevance between each scenario and each keyword in the keyword set for that topic; Based on the semantic relevance, the narrative function label for each scene is determined from the candidate set of narrative functions, so that the narrative function label for each scene is semantically associated with at least one topic keyword in the set of topic keywords.

[0182] For example, when determining the narrative function annotation for each scene, it is necessary to identify a candidate set of narrative functions for each scene based on the events it contains. This aims to comprehensively identify all potential narrative functions a scene may carry. For instance, by analyzing the types of events within a scene, the participating roles, the event outcomes, and the logical relationships between events, the possible narrative function labels for that scene can be matched or inferred from a predefined narrative function library. This can be achieved through semantic analysis of event descriptions using natural language processing techniques, or through event ontology mapping, rule-based reasoning, and other methods. For example, if a scene contains the event of "character A and character B arguing," its candidate set of narrative functions might include "conflict erupts," "relationship tension," and "changes in character emotions."

[0183] It is necessary to determine the semantic relevance of each scene to the keywords in the set of keywords for that scene. This step is used to quantify the relevance between the scene content (or its narrative function candidate set) and the core theme of the story. For example, if the set of keywords contains "growth" and "courage," and the event description of a scene or its narrative function candidate set contains semantics such as "overcoming difficulties" and "self-breakthrough," then the scene will have a high semantic relevance to the keywords such as "growth" and "courage." Semantic relevance can be obtained by calculating the cosine similarity between the word vectors or sentence vectors of the scene text description, event description, or narrative function candidate set and the keywords. In addition, knowledge graphs or semantic networks can also be used to measure the distance or strength of association between concepts.

[0184] Based on the semantic relevance, the narrative function annotation for each scene is determined from the candidate set of narrative functions, ensuring that the narrative function annotation for each scene establishes a semantic association with at least one theme keyword in the theme keyword set. After obtaining the candidate set of narrative functions for a scene and its semantic relevance with the theme keywords, the system will select narrative functions that represent the narrative intent of the scene and are relevant to the theme from the candidate set according to a preset filtering strategy. For example, a semantic relevance threshold can be set, selecting only candidate functions with a relevance higher than the threshold; or the candidate functions can be sorted, selecting the several with the highest relevance. The key is that the selected narrative function annotation must have a clear semantic connection with at least one keyword in the theme keyword set to ensure that the narrative function of the scene always serves the overall theme of the story.

[0185] By analyzing the events contained in each scene, the aforementioned technical solution comprehensively identifies the potential narrative functions of that scene, forming a candidate set of narrative functions and avoiding the omission of important narrative possibilities. Quantifying the semantic correlation between each scene and the set of thematic keywords provides an objective basis for subsequent narrative function selection, ensuring the consistency between the scene's narrative function and the overall story theme. Based on this semantic correlation, narrative function annotations that embody the theme are precisely selected from the candidate set, and a semantic connection between these annotations and at least one thematic keyword is mandated, thus resolving the issue of narrative function annotations potentially deviating from the main theme or lacking precision. This makes the narrative function annotations for each scene more accurate and purposeful, ensuring the narrative coherence of the storyboard sequence and the clarity of thematic expression, providing a more solid and guiding foundation for subsequent shot decomposition, and ultimately improving the narrative quality and artistic expression of the generated storyboard.

[0186] While the above-mentioned implementations can generate scene sequences and narrative function annotations for each scene through hierarchical planning, and obtain a set of thematic keywords, effectively transforming these high-level narrative plans and thematic intentions into specific, actionable shot parameters and scene content descriptions to form a complete and expressive storyboard sequence remains a challenge. Traditional methods may struggle to ensure that the generated storyboard sequence not only conforms to the narrative function but also accurately reflects the narrative intentions mapped by the thematic keywords, leading to a disconnect between the storyboard and the overall narrative goal, affecting the quality and coherence of the visual presentation.

[0187] To address this, this application further proposes a method that combines narrative function annotation, a preset film grammar constraint library, a set of thematic keywords, and narrative intent parameters to decompose each scene in a scene sequence into shots, generating shooting parameters and image content descriptions for each shot, thus forming a storyboard sequence. Specifically, this method includes: retrieving matching shooting specifications from the preset film grammar constraint library based on narrative function annotation; mapping the set of thematic keywords to narrative intent parameters; and decomposing each scene in the scene sequence into shots based on the shooting specifications and narrative intent parameters to obtain a storyboard sequence, which includes shooting parameters and image content descriptions for each shot.

[0188] The process of retrieving matching shooting guidelines from a pre-built film grammar constraint library based on narrative function annotations involves retrieving and obtaining shooting guidelines that align with the narrative function of a scene based on its role in the overall story or the specific information it conveys (i.e., narrative function annotations, such as "conflict outbreak," "emotional turning point," "background setup," etc.). These shooting guidelines can include recommended shot combinations, camera movement patterns, editing rhythm, and composition principles. Retrieving matching shooting guidelines can be achieved through semantic matching, rule-based reasoning, or machine learning-based recommendation algorithms (such as TF-IDF or word vector-based cosine similarity retrieval algorithms, K-Nearest Neighbors (KNN) algorithms, etc.). For example, for a scene labeled "conflict outbreak," the system might retrieve shooting guidelines that emphasize tension and impact, such as using close-ups, tight shots, rapid editing, and handheld shooting to visually enhance the narrative effect.

[0189] Mapping a set of thematic keywords to narrative intent parameters refers to transforming the core themes and emotional tone of a story (i.e., the set of thematic keywords, such as "courage," "sadness," "hope," and "oppression") into a series of specific parameters (i.e., narrative intent parameters) used to guide the visual style and emotional expression of shots. These narrative intent parameters can include color tone, light and shadow contrast, compositional focus, camera movement speed, and depth of field selection. This mapping can be achieved through a pre-defined style dictionary, expert system rules, or deep learning-based style transfer models (such as CycleGAN or Style Transfer Network). For example, when the thematic keyword contains "sadness," the narrative intent parameters might be set to low saturation, cool colors, slow camera movement, and deep depth of field to create a heavy and oppressive atmosphere; when the thematic keyword contains "hope," it might be mapped to high brightness, warm colors, wide-angle shots, and shallow depth of field to express a positive and expansive emotion.

[0190] Based on shooting guidelines and narrative intent parameters, each scene in the scene sequence is broken down into shots to obtain a storyboard sequence. This storyboard sequence includes the shooting parameters and content description for each shot, meaning that a complete scene is decomposed into a series of continuous shots with specific visual expressiveness. Shooting parameters are the specific technical specifications of each shot, such as shot type (close-up, medium shot, long shot, etc.), shooting angle (eye-level, low-angle, high-angle, etc.), and camera movement (push, pull, pan, tilt, track, rise, fall, etc.). The content description provides textual or visual explanations of the elements, composition, character actions, expressions, props, etc., contained in each shot. During the shot decomposition process, shooting guidelines provide basic, structural guidance; for example, a dialogue scene may require shot-reverse-shot, medium shot establishment, etc. Narrative intent parameters then refine and stylize these basic guidelines. For example, shooting guidelines might suggest a medium shot, but if the narrative intent parameters indicate an intention to convey "oppression," the medium shot might be modified to a low-angle medium shot with a darker color tone. The generated storyboard sequence is a collection of shot units arranged in chronological order. Each shot unit contains its unique shooting parameters and description of the scene content, which together constitute the visual narrative of the scene.

[0191] The aforementioned technical solution tightly integrates high-level narrative planning with specific visual presentation, resolving the disconnect between storyboarding and overall narrative goals in traditional methods. For example, based on the narrative function annotations of scenes, the system can intelligently retrieve shooting specifications conforming to professional film production standards from a pre-set film grammar constraint library, ensuring that the storyboards conform to narrative logic in structure and rhythm, providing a solid foundation for subsequent shot decomposition. Simultaneously, by mapping the set of thematic keywords to narrative intent parameters, the visual style and emotional expression of the storyboards accurately convey the core theme and emotional tone of the story. Shooting specifications provide a basic shot decomposition framework and technical guidance, while narrative intent parameters are refined and stylized based on this, ensuring that the shooting parameters and content descriptions of each shot are not only technically feasible but also artistically expressive. This dual constraint mechanism ensures that the generated storyboard sequence not only fulfills narrative functions but also deeply aligns with the story's thematic intent, enhancing the automation and artistic quality of storyboard generation and guaranteeing the coherence and impact of the visual presentation.

[0192] In some of the embodiments described above in this application, when retrieving matching shooting specifications from a preset film grammar constraint library based on narrative function annotations, if only generalized matching is performed using narrative function annotations, the retrieved shooting specifications may not be accurate enough and may not effectively guide subsequent shot decomposition. In particular, in terms of ensuring visual coherence and narrative fluency between scenes, there may be problems with inappropriate selection of the starting and ending shot framing.

[0193] In response, this application further proposes a step for retrieving matching shooting specifications from a pre-defined film grammar constraint library based on the aforementioned narrative function annotations, including: Based on the narrative function annotation, determine the matching constraint template type; Retrieve the shooting specifications corresponding to the constraint template type from the preset film grammar constraint library; Based on the narrative function annotation, the shot size of the first shot and the shot size of the last shot of the scene are determined from the shooting specifications.

[0194] For example, when determining the matching constraint template type based on narrative function annotations, this step aims to transform abstract narrative function annotations into more specific and actionable film grammar constraint template types. Narrative function annotations may describe the macro-narrative purpose of a scene, such as "introducing a new character," "driving the plot forward," or "resolving conflict." To more effectively filter relevant specifications from the vast film grammar constraint library, the system maps these narrative function annotations to predefined, finer-grained constraint template types. For example, if the narrative function annotation is "introducing a new character," the matching constraint template type might be determined as "character introduction template"; if it is "climax conflict," it might be determined as "action climax template." This mapping can be implemented through a pre-set rule set, an expert knowledge-based mapping table, or a machine learning model (such as decision trees, random forests, Transformer-based text classification models, etc.), thereby providing more precise guidance for subsequent shooting specification retrieval.

[0195] When retrieving shooting guidelines corresponding to a given constraint template type from the preset film grammar constraint library, this step uses that type as a query condition after identifying the matching constraint template type. This preset film grammar constraint library is a structured knowledge base containing a large number of general rules, best practices, and scene-specific shooting guidelines for filmmaking. Each constraint template type is associated with one or more detailed shooting guidelines, which may cover aspects such as shot size, shooting angle, camera movement, and editing rhythm. Retrieving by precise constraint template type ensures that the retrieved shooting guidelines align with the narrative function of the current scene, avoiding interference from general or irrelevant guidelines, thereby improving the applicability and effectiveness of the shooting guidelines.

[0196] When determining the opening and closing shot sizes of a scene based on the narrative function annotations and shooting guidelines, this step focuses on ensuring visual continuity within and between scenes. After retrieving shooting guidelines corresponding to the constraint template type, the system further combines the scene's narrative function annotations to select suitable shot sizes for the scene's beginning and end. For example, if the narrative function annotation indicates that the scene is "setting the environment," its opening shot size might be determined as "long shot" or "wide shot"; if the narrative function annotation is "close-up of character's emotion," the closing shot size might be determined as "close-up" or "close-up." This precise determination of the opening and closing shot sizes based on narrative function annotations helps control the opening and closing of a scene, ensuring a natural visual connection with the preceding and following scenes, and effectively conveying the scene's narrative intent, thereby enhancing the narrative smoothness and visual impact of the overall storyboard sequence.

[0197] Through the above technical solution, when retrieving matching shooting specifications from a preset film grammar constraint library based on narrative function annotations, the matching constraint template type is determined according to the narrative function annotations. This makes the retrieval process of shooting specifications more focused and precise, avoiding the inefficiency and inaccuracy that may result from fuzzy matching directly from a large library. Retrieving shooting specifications corresponding to the constraint template type from the preset film grammar constraint library ensures that the obtained specifications match the narrative purpose of the scene. Determining the opening and closing shot sizes of a scene from the retrieved shooting specifications based on narrative function annotations not only ensures the integrity of the visual narrative within each scene but also solves the problem of unnatural transitions between scenes, making the storyboard sequence more visually coherent and narratively smoother, thereby improving the professionalism and artistic expression of the generated storyboard results.

[0198] In some embodiments described above in this application, a set of thematic keywords is mapped to narrative intent parameters to guide the shot breakdown of a scene. However, if this mapping process fails to adequately consider the unique narrative function of each scene, the generated narrative intent parameters may become overly generalized, failing to provide precise visual guidance for a specific scene, thereby affecting the effectiveness of the shot sequence in expressing the scene's narrative intent.

[0199] In response, this application further proposes a specific method for mapping the aforementioned set of thematic keywords to narrative intent parameters. This method includes: determining target thematic keywords from the set that are semantically related to the narrative function annotation of each scene; determining the shot composition orientation of each scene based on the target thematic keywords; and determining the shot composition orientation as the narrative intent parameter.

[0200] For example, when determining target keywords semantically related to the narrative function annotations of each scene, this step aims to filter out more targeted keywords directly related to the narrative function annotations of the current processing scene from a macro-level keyword set. Natural language processing techniques can be used to calculate the semantic similarity between the narrative function annotations of each scene and each keyword in the keyword set. For instance, word vector models (such as Word2Vec, BERT, etc.) can be used to convert the narrative function annotations and keyword tags into high-dimensional vector representations, and then the cosine similarity between these vectors can be calculated. When the similarity exceeds a preset threshold, the keyword is identified as a target keyword semantically related to the current scene. This ensures that subsequent visual composition guidance closely revolves around the core narrative intent of the scene.

[0201] Based on the target theme keywords, determine the shot composition orientation for each scene. This step transforms the abstract narrative theme into a concrete visual expression. Shot composition orientation refers to the overall style and tendency guiding the visual presentation of a scene, such as emphasizing a sense of vast space, highlighting character emotions, creating a tense atmosphere, or showcasing details. A knowledge base or rule set can be pre-built to associate different target theme keywords (such as "oppression," "hope," and "conflict") with a series of visual composition elements (such as shot size preference, color tendency, lighting usage, and compositional lines). For example, when the target theme keyword is "oppression," the corresponding composition orientation might be low saturation, dark lighting, and closed composition; when it is "hope," it might correspond to highlights, warm colors, and open composition. By consulting or reasoning from this knowledge base, the unique shot composition orientation for each scene can be determined.

[0202] The shot composition orientation is then defined as the narrative intent parameter. This step involves structuring and parameterizing the aforementioned visual composition guidelines to facilitate analysis and application in the subsequent shot breakdown module. The narrative intent parameter can be a data structure containing multiple fields, each representing a specific visual guidance dimension, such as shot type preference (e.g., "primarily close-up," "primarily wide shot"), color style (e.g., "cool tones," "warm tones"), lighting mode (e.g., "high contrast," "soft light"), and center of gravity (e.g., "centered," "offset"). Filling these parameter fields with the various visual elements from the shot composition orientation creates the scene-specific narrative intent parameter, providing clear and actionable input for subsequent shot breakdown.

[0203] The above technical solution ensures that the narrative intent parameters extracted from the set of thematic keywords align with the narrative function annotations of each scene. By identifying target thematic keywords semantically related to the scene, and then determining the scene's shot composition orientation based on these keywords, this orientation is transformed into narrative intent parameters. This ensures that the visual expression of each scene serves its specific narrative purpose and overall theme. This avoids the visual ambiguity or deviation that may result from generalized parameters, improves the accuracy and impact of the storyboard sequence in visual storytelling, and enables the generated storyboard sequence to more effectively convey the deeper meaning and emotions of the story.

[0204] In some of the embodiments described above in this application, the system has completed semantic parsing and hierarchical planning of the narrative text, and generated scene sequences and narrative function annotations. It has also retrieved matching shooting specifications from a preset film grammar constraint library and mapped narrative intent parameters. However, when converting these high-level plans and parameters into specific executable storyboard sequences, a lack of effective control over the visual coherence between shots—for example, overly abrupt changes in shot size between adjacent shots—may lead to insufficient visual smoothness in the storyboard sequence, affecting the viewer's viewing experience and understanding of the narrative content.

[0205] In response, this application further proposes to decompose each scene in the above scene sequence into a shot sequence based on the above shooting specifications and the above narrative intent parameters. This process includes: decomposing each scene into a shot sequence based on the above shooting specifications and the above narrative intent parameters; determining the shot size, shooting angle, and camera movement for each shot in the above shot sequence as the above shooting parameters, wherein the difference in shot size between adjacent shots does not exceed a preset threshold; generating a scene content description for each shot; and organizing the above shooting parameters and the above scene content description in scene order to form the above shot sequence.

[0206] For example, based on the aforementioned shooting guidelines and narrative intent parameters, decomposing each scene into a sequence of shots means refining a macro-level scene concept into a series of specific, continuous shots. This step is the core of storyboard generation, determining the scene's narrative rhythm, information delivery method, and overall visual style. In implementation, a pre-defined shot decomposition template library can be used. These templates offer multiple decomposition schemes based on different scene types (such as dialogue, action, emotional expression) and narrative functions (such as setup, climax, and transition). Simultaneously, combining the general film grammar rules regarding shot composition in the aforementioned shooting guidelines with the personalized guidance in the aforementioned narrative intent parameters for specific narrative goals (such as emphasizing a character or creating a specific emotion), the system can intelligently select or generate appropriate shot sequences. For instance, for a scene depicting a "tense chase," the shooting guidelines might suggest using fast cuts and multi-angle transitions, while the narrative intent parameters might further refine this to using handheld shooting and low-angle shots to enhance tension.

[0207] For each shot in the aforementioned shot sequence, a shot size, shooting angle, and camera movement are determined as shooting parameters. The requirement that the difference in shot size between adjacent shots does not exceed a preset threshold means assigning specific visual expression instructions to each independent shot. Shot size (e.g., long shot, full shot, medium shot, close-up, extreme close-up) determines the amount of information contained in the image and the psychological distance between the viewer and the content; shooting angle (e.g., eye-level, low-angle, high-angle) influences the viewer's emotional inclination and perspective on the content; camera movement (e.g., push, pull, pan, tilt, track) gives the image a sense of dynamism, guides the viewer's gaze, and affects the narrative rhythm. When determining these parameters, the system comprehensively considers the narrative function of the scene, emotional goals, elements to be emphasized in the image, and narrative intent parameters. For example, when depicting a character's inner struggle, a extreme close-up shot and a slightly high-angle angle might be chosen; when showcasing a vast scene, a long shot combined with a panning shot might be selected. Specifically, to ensure visual smoothness in shot transitions and avoid a sense of jarring visuals, this application introduces the constraint that "the difference in shot size between adjacent shots does not exceed a preset threshold." This means that in the shot scale rating table, the change in shot scale between adjacent shots cannot be too abrupt. For example, if the preset threshold is 1, it is not possible to jump directly from a long shot to a close-up, but a transition through a medium shot or close-up is required to ensure a smooth visual transition and the continuity of the narrative rhythm.

[0208] Generating a scene description for each shot involves transforming abstract shooting parameters and narrative intent into concrete, visually achievable scene content instructions. This description details the elements that should be included in the scene (such as characters, props, and environment), their layout, actions, expressions, and lighting effects. During generation, the system comprehensively considers each shot's shooting parameters (such as the framing determining the focus of the scene content), the scene's narrative elements (such as character entities, scene entities, and events), and narrative intent parameters (such as emphasizing a character's emotion). For example, for a close-up shot expressing sadness, the scene description might specifically state, "Close-up of the character's face, tears in their eyes, a sad expression, and a blurred background."

[0209] Organizing the aforementioned shooting parameters and scene descriptions into the storyboard sequence according to scene order refers to the process of structuring and serializing all generated shot information. Each shot unit contains its corresponding shooting parameters (shot size, shooting angle, camera movement) and scene description. These shot units are arranged according to their logical order within their respective scenes, and the shot sequences of each scene are then organized according to their chronological order within the entire narrative, forming a complete and orderly storyboard sequence. This storyboard sequence can serve as a detailed guide for subsequent animation, filming, or live-action shooting.

[0210] The aforementioned technical solution meticulously transforms high-level narrative planning and cinematic grammar rules into concrete, executable storyboard sequences. In particular, by defining detailed shooting parameters for each shot and introducing the constraint that "the difference in shot size between adjacent shots does not exceed a preset threshold," the potential visual abruptness of shot transitions within the storyboard sequence is resolved, ensuring visual coherence and smoothness. Simultaneously, generating clear descriptions of the scene content for each shot provides explicit guidance for subsequent visualization work, improving the usability and accuracy of the generated storyboard results. Organizing these detailed shooting parameters and scene content descriptions into a storyboard sequence according to scene order makes the transformation of the entire narrative flow from abstract planning to concrete visual presentation more complete and efficient, thereby enhancing the practical value and user experience of the storyboard generation method.

[0211] In some of the above implementation methods, although the scene can be decomposed into a sequence of shots based on shooting specifications and narrative intent parameters, in actual operation, if the shot decomposition strategy is determined solely by the preset shooting specifications, it may not be able to fully reflect the refined requirements of the specific narrative intent on the shot language, resulting in the generated shot sequence lacking flexibility and depth in expressing the narrative intent.

[0212] In response, this application further proposes a specific method for decomposing each scene into a sequence of shots based on shooting specifications and narrative intent parameters, which includes: determining the shot decomposition strategy for each scene based on shooting specifications; constraining and modifying the shot decomposition strategy based on narrative intent parameters; and decomposing each scene into a sequence of shots based on the modified shot decomposition strategy.

[0213] For example, when generating a storyboard sequence, the shot breakdown strategy for each scene needs to be determined based on shooting guidelines. These guidelines are a set of long-standing principles and conventions in filmmaking regarding shot usage, combination, and transitions, designed to ensure the coherence and effectiveness of the visual narrative. The shot breakdown strategy, based on these guidelines, breaks down a complete scene into a series of independent shots. For instance, it might specify the number and type of shots (e.g., wide shot, medium shot, close-up), transition rhythm, and approximate shooting angles. This strategy can be determined by consulting a pre-defined film grammar template library, using expert system rule reasoning, or learning from large amounts of film data using machine learning models (e.g., decision trees, random forests, graph neural networks (GNNs)). For example, for a scene depicting a character's inner monologue, the shooting guidelines might recommend more close-ups and slow camera movements; while for an action chase scene, it might favor rapidly switching medium and close-up shots.

[0214] To enhance the expressiveness of the generated shot sequences and accurately convey narrative intent, this application further constrains and modifies the shot decomposition strategy based on narrative intent parameters. These parameters, mapped from a set of thematic keywords, embody the deeper meaning, emotional tone, visual style, or narrative focus the creator wishes to express through visual language. Constraining and modifying the shot decomposition strategy means adjusting and optimizing the initially determined general decomposition strategy based on these personalized narrative intents. For example, if the narrative intent parameters indicate that a scene needs to emphasize "oppression," the modification process might adjust the strategy, increasing the proportion of close-up shots taken from low angles and reducing open compositions; if the narrative intent parameters indicate "grand narrative," it might increase wide shots or panoramic shots and employ smooth push-pull camera movements. This modification can be achieved through a pre-defined mapping rule library, transforming specific narrative intent parameters into concrete modification instructions for aspects such as the number of shots, shot size distribution, shooting angle preferences, and camera movement intensity; or it can be achieved through optimization algorithms (such as genetic algorithms, simulated annealing algorithms, particle swarm optimization (PSO) algorithms, etc.) to maximize the expressive effect of the narrative intent while meeting basic shooting specifications.

[0215] Based on the revised shot decomposition strategy, each scene is broken down into a sequence of shots. This revised strategy, refined with narrative intent parameters, has become more sophisticated and personalized, better serving specific narrative objectives. The system will strictly adhere to this revised strategy to generate a detailed list of shots, including preliminary shot size, shooting angle, and motion parameters for each shot. This process ensures that the generated shot sequence not only structurally conforms to conventional filmmaking requirements but also deeply integrates the creator's unique narrative intent in both content and form, thus laying a solid foundation for subsequent shooting parameter determination and image content description generation.

[0216] Through the aforementioned technical solution, a preliminary shot breakdown strategy is determined based on general shooting guidelines, ensuring that the shot sequence conforms to basic film grammar rules. Narrative intent parameters are introduced to constrain and modify this strategy, enabling the shot breakdown process to deeply integrate the creator's narrative intent and emotional expression needs. This modification mechanism solves the problem that relying solely on general guidelines may lead to a lack of personalization and expressiveness in the shot sequences, ensuring that the generated shot sequences are not only structurally sound but also precisely targeted in emotional and thematic expression. It allows the camera language of each scene to serve the overall narrative more flexibly and subtly, thereby generating shot sequences with greater artistic impact and narrative depth.

[0217] In some of the embodiments described above in this application, although a shot decomposition strategy for each scene is proposed based on shooting specifications, and this strategy is further constrained and modified based on narrative intent parameters to achieve shot decomposition of the scene, the lack of a clear specific modification mechanism may result in a lack of refined guidance in the modification process of the shot decomposition strategy. This makes it impossible for the generated shot sequence to accurately reflect the narrative intent in terms of shot type and duration allocation, thereby affecting the overall expressiveness and narrative appeal of the storyboard sequence.

[0218] In response, this application further proposes a step of constraining and modifying the shot decomposition strategy based on the narrative intent parameter, including: determining the shot type allocation scheme in the shot decomposition strategy based on the narrative intent parameter; determining the shot duration allocation scheme in the shot decomposition strategy based on the narrative intent parameter; and modifying the shot decomposition strategy based on the shot type allocation scheme and the shot duration allocation scheme to obtain the modified shot decomposition strategy.

[0219] For example, determining the shot type allocation scheme in this shot decomposition strategy refers to how to select appropriate shot types, such as close-ups, medium shots, long shots, and extreme shots, when decomposing a scene into multiple shots. Narrative intent parameters, derived from a mapping of thematic keyword sets, provide deeper information about the scene's core theme, emotional tone, and visual style. For instance, when the narrative intent parameters indicate that a scene needs to emphasize the character's inner world or detailed reactions, the system will tend to increase the proportion of close-ups or medium shots in the shot type allocation scheme; conversely, if it needs to showcase a grand environmental background or the interaction between characters and their environment, it will tend to allocate more long shots or extreme shots. This scheme can be determined through a pre-defined mapping rule base, machine learning-based models (such as decision trees or neural networks), or expert systems. These systems can output a specific scheme containing different shot types and their proportions or timing in the shot sequence based on the input narrative intent parameters.

[0220] Simultaneously, determining the shot duration allocation scheme in this shot decomposition strategy refers to how to allocate the duration of each shot in the generated shot sequence. Narrative intent parameters also play a crucial role in this stage. For example, if the narrative intent parameters indicate that a scene needs to create a tense, fast-paced narrative rhythm, the shot duration allocation scheme will tend to allocate shorter shot durations to achieve a fast-paced and compact effect; conversely, if a soothing, contemplative, or solemn atmosphere is desired, it will tend to allocate longer shot durations to give the audience ample time to observe and experience. This scheme can be determined based on a pre-defined rhythm rule library, dynamic adjustment algorithms (such as proportional control algorithms, sliding window averaging algorithms, or gradient descent-based iterative duration optimization algorithms), or by combining calculations from a narrative rhythm model. Based on the narrative intent parameters, a specific scheme describing the average duration of each shot or different types of shots is generated.

[0221] Modifying the shot decomposition strategy refers to the overall plan that guides how a scene is broken down into a series of shots. This plan may include the initial number of shots, the general trend of shot size changes, etc. Using the determined shot type allocation scheme and shot duration allocation scheme, this initial strategy can be finely adjusted and optimized. For example, if the initial strategy plans a specific number of shots, but the shot type allocation scheme requires increasing the proportion of a certain type of shot, it may be necessary to adjust the types of existing shots, or adjust the shot size of some shots without changing the total number of shots. Simultaneously, the shot duration allocation scheme guides the specific duration of each shot, thus directly affecting the narrative rhythm of the entire scene. The modification process can be an iterative optimization process, ensuring that the shot decomposition strategy both conforms to film grammar and accurately expresses the narrative intent. This may involve weight adjustments, priority ranking, or strategy reconstruction based on optimization algorithms (such as genetic algorithms, simulated annealing algorithms, gradient descent algorithms, etc.) to achieve the desired narrative effect.

[0222] The aforementioned technical solution enables refined constraints and corrections to the shot decomposition strategy based on narrative intent parameters. For example, by clearly defining the shot type allocation scheme and shot duration allocation scheme, this application allows for precise implementation of the core theme, emotional tone, and rhythmic requirements of the scene implied by the narrative intent parameters into the shot size and duration of each shot when decomposing a scene into a shot sequence. For instance, when the narrative intent emphasizes tension, the system tends to allocate more short close-up shots; when a grand narrative needs to be presented, it tends to allocate longer panoramic shots. This refined correction mechanism ensures that the generated shot sequence is consistent with the overall narrative theme in terms of visual expression and narrative rhythm, enhancing the artistic expression and narrative appeal of the shot sequence, avoiding the generation of generic or inconsistent shot combinations, thus making the generated shot sequence more targeted and expressive.

[0223] In some of the embodiments described above in this application, although it is proposed to determine the shot size, shooting angle and camera movement as shooting parameters for each shot in the shot sequence, and require that the difference in shot size between adjacent shots does not exceed a preset threshold, in actual implementation, how to efficiently and accurately select shooting parameters from a variety of possible combinations of shooting parameters that not only conform to the norms of film grammar, can accurately express the narrative intent, and also meet the constraints of smooth transition between adjacent shots is a key challenge to ensure the quality and coherence of the shot sequence.

[0224] In response, this application further proposes to determine the shot size, shooting angle, and camera movement mode for each shot in the shot sequence as shooting parameters, including: determining a set of candidate shooting parameters for each shot according to the shooting specifications, the set of candidate shooting parameters including candidate shot size, candidate shooting angle, and candidate camera movement mode; selecting initial shooting parameters for each shot from the set of candidate shooting parameters according to the narrative intent parameter; and correcting the shot size in the initial shooting parameters of each shot according to the shot size level difference constraint between adjacent shots to obtain the shooting parameters.

[0225] To address this, this application determines a set of candidate shooting parameters for each shot based on the shooting specifications. These shooting specifications are retrieved from a pre-defined film grammar constraint library based on the narrative function annotations of the current scene, and contain recommended or permitted shot language elements for specific narrative functions. For example, for a scene depicting a character's inner monologue, the shooting specifications might recommend using close-ups or medium shots to focus on the character's expression; recommend using low angles or eye-level angles to enhance immersion; and recommend using fixed shots or slow push-in / pull-out shots to create a tranquil atmosphere. Based on these specifications, the system generates a candidate set containing various combinations of shot sizes, shooting angles, and camera movements, ensuring that the selected parameters conform to basic film grammar rules and scene type requirements.

[0226] This application further filters the initial shooting parameters for each shot from the candidate shooting parameter set based on the narrative intent parameter. This narrative intent parameter is mapped from a set of thematic keywords, reflecting the core theme and emotional tendency the scene intends to convey. Within the candidate shooting parameter set, there may be multiple options that conform to shooting specifications, but not all options better serve the narrative intent of the current scene. For example, if the narrative intent parameter emphasizes "oppression" or "insignificance," then in the candidate shot types, a long shot or a wide shot may better reflect this intent than a close-up; if it emphasizes "tension," then rapid push-pull or handheld shaky shots may be more appropriate. By semantically matching or prioritizing the narrative intent parameter with the candidate shooting parameters, the system can select the shot type, shooting angle, and camera movement that strengthen the intent as the initial shooting parameters for the shot, thereby ensuring that the visual expression of the storyboard aligns with the narrative content.

[0227] To ensure the coherence and visual smoothness of the storyboard sequence, this application, after determining the initial shooting parameters, further modifies the shot size in the initial shooting parameters of each shot based on the constraint of the shot size difference between adjacent shots, thus obtaining the final shooting parameters. Film grammar typically requires that the shot size changes between adjacent shots not be too drastic to avoid visual jumps or narrative interruptions for the audience. The system traverses the entire shot sequence, checking whether the shot size difference between adjacent shots exceeds a preset threshold. For example, if the initial shooting parameters cause a jump directly from a close-up shot to a long shot, and this jump exceeds the allowed shot size difference threshold, the system will intervene to correct it. Correction strategies could include adjusting the shot size of one shot to be closer to that of the adjacent shot, or inserting a transition shot size between them to achieve a smooth transition. This correction mechanism ensures the visual coherence and professionalism of the storyboard sequence, avoiding visual discomfort caused by abrupt changes in shot size.

[0228] The aforementioned technical solution generates a set of candidate shooting parameters that conform to cinematic grammar rules for each shot, ensuring basic professionalism and diversity. Further, by combining narrative intent parameters, initial shooting parameters that reflect the core theme and emotional tone of the scene are selected from these candidate parameters, thus ensuring that the visual expression of the storyboard aligns with the narrative content. By rigorously verifying and correcting the difference in shot size between adjacent shots, visual jumps and inconsistencies are effectively avoided, guaranteeing the smoothness and professionalism of the storyboard sequence. This results in a storyboard sequence that is not only more profound in artistic expression but also more natural and harmonious in visual presentation, improving the automation level and quality of storyboard generation.

[0229] In some of the embodiments described above in this application, although the shot size, shooting angle, and camera movement have been proposed as shooting parameters for each shot in the shot sequence, how to systematically and accurately generate a description of the scene content that matches the intent of the shot based on these determined shooting parameters, so as to ensure the visual coherence and narrative accuracy of the storyboard sequence, remains a problem that needs to be refined. If the generation process of the scene content description lacks clear guidance and structure, it may lead to a disconnect from the shooting parameters, resulting in a discrepancy between the visual presentation of the storyboard and the expected narrative effect, affecting the quality of the work and production efficiency.

[0230] In response, this application further proposes a method for generating a scene content description for each shot, including: determining the scene composition frame of each shot based on the shooting parameters of each shot; determining the scene element set of each shot based on the scene composition frame; and generating the scene content description of each shot based on the scene element set.

[0231] For example, when determining the composition framework for each shot based on its specific shooting parameters, these parameters—such as framing, shooting angle, and camera movement—directly determine the basic structure and visual presentation of the image. The composition framework is a preliminary plan based on these parameters, outlining the spatial arrangement of the subject, the position of the subject, and visual leading lines. For instance, when the framing is a "close-up," the composition framework might emphasize the subject's face or specific details, potentially indicating that the subject should occupy a slightly above-center position in the frame. When the shooting angle is a "short shot," the framework might emphasize the subject's height or authority by shooting upwards from a low angle. When the camera movement is a "push-in," the framework pre-sets a visual path that gradually focuses from a broad scene to a specific detail. This can be achieved through a pre-defined composition rule base or machine learning-based models (such as decision trees, random forests, and convolutional neural networks (CNNs)) that map different combinations of shooting parameters to specific composition templates or guidelines, thus providing structured guidance for subsequent content filling.

[0232] Based on the compositional framework, the set of visual elements for each shot is determined. The compositional framework provides the skeleton of the image, while the set of visual elements is the specific visual content filled into this skeleton, including but not limited to characters, objects, backgrounds, lighting, colors, and props. For example, if the compositional framework indicates that the main subject is on the left and the scene is set in a library, the set of visual elements would include "main character A on the left side of the frame," "bookshelf," "books," and "reader." Furthermore, determining the set of visual elements also requires combining the original set of narrative elements (such as character entities, scene entities, and event sequences) and the set of thematic keywords to ensure that the content of the image is consistent with the story development and narrative theme. For example, if the thematic keyword includes "loneliness," the set of visual elements might be further refined to include "empty room," "single light source," and "character's back view" to create a specific atmosphere. This can be achieved through semantic matching, knowledge graph retrieval, or pre-trained visual element generation models, intelligently selecting and combining visual elements based on the compositional framework and narrative context.

[0233] Based on the set of visual elements, a description of the content of each shot is generated. This step organizes the defined set of visual elements into a clear and specific text description in natural language for use in subsequent visualization or production stages. For example, a preset description template can be used to fill in the various contents of the set of visual elements, such as: "[Shot type] Shot, [Shooting angle] Shooting [subject], [Subject action], [Background description], [Lighting / color description]." Alternatively, a Natural Language Generation (NLG) model can be used to generate more expressive and diverse descriptive text based on the set of visual elements and contextual information. The description should include sufficient detail, such as the character's expression, actions, and clothing; the material, color, and position of objects; and the atmosphere and lighting of the environment, while remaining concise and clear. For example, instead of simply saying "a person," it should say "a man wearing a gray trench coat, with a melancholy expression, stands by the window, gazing into the distance."

[0234] The aforementioned technical solution allows for the systematic derivation of the compositional framework for each shot based on predetermined shooting parameters. This framework then determines the set of visual elements and generates a narrative description that matches the shooting parameters. This resolves the potential disconnect between the narrative description and shooting parameters during storyboard generation, ensuring visual coherence and accurate communication of the narrative intent. This method provides a clear and operational workflow, making the generation of narrative descriptions no longer arbitrary or subjective, but based on explicit rules and inputs. This improves the efficiency and quality of storyboard production and provides a solid and precise foundation for subsequent visualization work.

[0235] In some of the embodiments described above in this application, although a technical solution has been proposed to decompose each scene in a scene sequence into shots, generate shooting parameters and image content descriptions for each shot, and form a storyboard sequence, in practical applications, simply generating these independent shooting parameters and image content descriptions cannot directly form a complete, easy-to-understand, and usable storyboard sequence for subsequent production. This scattered information requires a systematic organization to ensure its inherent logical coherence and narrative flow, thereby avoiding problems such as difficulty in information interpretation or workflow disruptions in subsequent production stages.

[0236] To address this, this application further proposes a method for organizing the shooting parameters and the scene content description into a storyboard sequence according to scene order. For example, the method includes the following steps: By associating the shooting parameters of each shot with the corresponding scene content description, a shot unit is obtained. In this step, for each generated shot, its corresponding shooting parameters (such as framing, shooting angle, camera movement, and other technical specifications) and scene content description (such as visual information such as characters, objects, background, and actions within the scene) are logically or physically bound together, forming a complete and self-consistent "shot unit." This association can be achieved through data structures, such as encapsulating the shooting parameters and scene content description in a data object or storing them in key-value pairs, ensuring that all relevant information for a single shot is concentrated in one entity, avoiding information dispersion and omission. This integrated shot unit is the foundation for constructing a coherent sequence of scenes.

[0237] Based on the position of each shot within the scene sequence, the order of each shot unit is determined. After generating multiple shot units, their correct sequence needs to be determined according to their position within the overall narrative structure. This includes identifying the scene to which each shot unit belongs and its order within that scene. By referring to a pre-planned scene sequence and the shot decomposition order within each scene, a global and precise temporal and narrative order can be established for all shot units. For example, if the scene sequence contains scenes A, B, and C, and scene A is decomposed into shots A1 and A2, and scene B is decomposed into shots B1 and B2, then the order would be A1, A2, B1, B2. This step ensures the narrative logic and visual coherence of the storyboard sequence, enabling it to accurately reflect the intent of the original narrative text.

[0238] Organizing the shot units according to this arrangement order yields the storyboard sequence. After determining the precise order of all shot units, these ordered shot units are integrated and assembled to form the complete storyboard sequence. This organizational process links the shot units in their defined order, generating a structured output, such as a digital file (e.g., XML or JSON format) containing information about all shot units, or a visually presented storyboard. This storyboard sequence not only includes detailed shooting parameters and descriptions of the scene content for each shot, but also presents it in a way that conforms to narrative logic and chronological order, providing clear and comprehensive guidance for subsequent animation production, live-action shooting, or post-production editing.

[0239] The above technical solution solves the problem of organizing scattered shot information into a coherent, user-friendly, and narratively logical complete shot sequence during the storyboard generation process. By closely linking the shooting parameters of each shot with the description of the scene content to form shot units, the integrity and self-consistency of individual shot information are ensured. Determining the arrangement order of shots based on their precise positions in the scene sequence guarantees the consistency of the entire storyboard sequence in terms of time and narrative logic. Organizing the shot units according to the determined arrangement order ensures that the generated storyboard sequence not only includes all necessary visual and technical guidance but also presents it in a structured and easy-to-understand manner, improving the practicality and operability of the storyboard sequence. This provides a clear and efficient blueprint for subsequent film and television production processes, thereby improving production efficiency and the quality of the final product.

[0240] The following example will provide a more detailed explanation of the above technical solution: For example, there is an input narrative text that describes a story about "courage" and "growth": a young explorer encounters difficulties in ancient ruins, overcomes the challenges with wisdom and perseverance, obtains the treasure, and returns home.

[0241] The input narrative text undergoes semantic parsing to obtain a set of narrative elements containing event sequences and character relationships. For example, the system identifies "young explorer" as a character entity and "ancient ruins" as a scene entity, extracting a series of events such as "explorer enters the ruins," "encounters traps," "gets into trouble," "thinks of countermeasures," "successfully solves the traps," "obtains treasure," and "safely leaves the ruins," forming an event sequence. Simultaneously, based on the interactions between character entities within these events, character relationships are determined, such as the antagonistic relationship between the explorer and the ruins environment. This process, through sentence-level segmentation and encoding of the text, and the identification of entities and events, ensures a comprehensive and structured understanding of the original narrative content, avoiding the loss of semantic information that might occur with direct mapping in related technologies.

[0242] The system performs first-level planning on the aforementioned set of narrative elements. It identifies key events from the event sequence, such as "getting into trouble" and "successfully solving a puzzle," and generates a narrative theme based on these key events, such as "achieving self-growth by overcoming challenges." Simultaneously, based on the event type distribution (such as tension, resolution, and relaxation) of each event in the sequence, it generates an emotional arc, for example, from calm to tension, then to hope, culminating in the joy of victory. Following preset refinement rules, the narrative theme is transformed into a set of thematic keywords, such as "courage," "wisdom," "growth," and "exploration," and the emotional target values ​​for each stage of the emotional arc are converted into corresponding numerical ranges, combined chronologically to form emotional range constraints. This hierarchical planning step establishes the core narrative intent and emotional trajectory at a macro level, providing clear high-level constraints for subsequent scene and shot generation, and solving the problem in related technologies where macro narrative intent cannot effectively constrain low-level composition selection.

[0243] Using a set of thematic keywords and emotional range constraints as input constraints, the system performs second-level planning on the event sequence within the narrative element set, resulting in a scene sequence and narrative function annotations for each scene. For example, the event sequence might be divided into scenes such as "Entering Ruins," "Encountering Challenges," "Solving Difficulties," and "Returning in Triumph." During scene division, the system ensures that scene boundaries do not interrupt the causal chain of events; for example, the causally related events of "Encountering a Trap" and "Getting into a Difficulty" are not separated by different scene boundaries. Simultaneously, the system assigns emotional change goals to each scene based on emotional range constraints; for example, the emotional goal for the "Encountering Challenges" scene is an increase in tension. Based on the events and thematic keyword set contained in each scene, the system determines the narrative function annotation for each scene; for example, the narrative function annotation for the "Encountering Challenges" scene is "Showing the Difficulty and the Explorer's Courage." This scene-level planning ensures that each scene carries a clear narrative function and emotional goal, and establishes a semantic connection with the overall narrative theme (through thematic keywords), further refining the constraints of high-level planning and providing mid-level guidance for subsequent shot decomposition.

[0244] Based on narrative function annotations, the system retrieves matching shooting guidelines from a pre-defined film grammar constraint library. These guidelines, combined with narrative intent parameters mapped from the theme keyword set, work together to decompose each scene in the scene sequence into shots, generating shooting parameters and content descriptions for each shot, thus forming a storyboard sequence. For example, for the narrative function annotation "showing the predicament and the explorer's courage," the system will retrieve shooting guidelines from the film grammar constraint library such as "close-ups to express character emotions" and "low-angle shots to enhance the sense of oppression." Simultaneously, the theme keyword "courage" may be mapped to the narrative intent parameter "emphasizing the character's subjectivity." During shot decomposition, the system breaks down the scene into a series of shots based on these shooting guidelines and narrative intent parameters. For example: a long shot shows the grandeur of the ruins and the explorer's insignificance; a medium shot shows the explorer's hesitation in the face of a trap; a close-up focuses on the explorer's resolute gaze; and a low-angle shot shows the moment the explorer successfully solves the trap. Each shot generates specific shooting parameters (such as shot size, shooting angle, and camera movement) and content descriptions. When determining the shot size of adjacent shots, the system adheres to a constraint that the difference in shot size level does not exceed a preset threshold to ensure the fluency of the shot language. This process uses film grammar rules as internal constraint parameters, combined with narrative intent parameters, to ensure that the generated shot sequence achieves professional standards in both narrative consistency and visual language standardization. This solves the problem in related technologies where film grammar rules are only used as external checking conditions, resulting in a lack of professional standardization in shot language. Through this progressive and interlocking planning and constraint mechanism, this solution can generate shot sequences that are consistent with the original narrative intent. Figure 1 A storyboard sequence that adheres to professional standards in visual language.

[0245] In this embodiment, the emotional arc is divided into four emotional stages: introduction, development, transition, and conclusion. The emotional target value for each stage is converted into a corresponding numerical range. For example, the emotional target value for the "introduction" stage is 0.2, and the corresponding numerical range is [0.1, 0.3]; the emotional target value for the "transition" stage is -0.3, and the corresponding numerical range is [-0.5, -0.1]. The "introduction" stage includes events such as "entering the ruins" and "thinking about countermeasures," and the emotional polarity scores for each event are 0.3, 0.3, and 0, respectively, with an average of 0. Therefore, the emotional target value for the "beginning" stage is set at 0.2. Expanding with 0.2 as the center and 0.1 as the half-width, the numerical range is [0.1, 0.3]. The "transition" stage includes negative and conflicting events such as "encountering traps" and "falling into difficulties." The emotional polarity scores for each event are -0.3, -0.3, and -0.3, respectively, with an average of -0.3. Therefore, the emotional target value for the "transition" stage is set at -0.3. Expanding with -0.3 as the center and 0.2 as the half-width, the numerical range is [-0.5, -0.1]. This forms an emotional range constraint covering the entire narrative timeline, with the emotional change targets for each scene falling within the corresponding numerical range. During the shot decomposition process, the difference in shot size between adjacent shots does not exceed a preset threshold. For example, when the preset threshold is 1, adjacent shots cannot directly switch from a long shot to a close-up, ensuring the visual smoothness of the shot language. Furthermore, "encountering a trap" and "falling into a predicament" form a causal chain, and both are kept within the same scene when dividing the scene, ensuring that the scene boundary does not interrupt the causal chain of events.

[0246] In this embodiment, the preset influence threshold is set to 0.5. Events with an influence exceeding 0.5 corresponding to the causal propagation depth in the event sequence are identified as candidate key events. The semantic relevance threshold is set to 0.6. The narrative function annotation of each scene is semantically associated with the theme keywords with a semantic relevance of not less than 0.6. The shot size is divided into five levels: long shot, full shot, medium shot, close-up, and extreme close-up. The difference in shot size level between adjacent shots does not exceed 1 to ensure visually smooth shot transitions.

[0247] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0248] The above are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A storyboard generation method based on hierarchical narrative planning and multidimensional constraint feedback, characterized in that, The method includes: Semantic parsing is performed on the input narrative text to obtain a set of narrative elements containing event sequences and character relationships; The narrative element set is planned at the first level to generate an emotional arc and narrative theme. Based on the preset refinement rules, the narrative theme is transformed into a set of theme keywords, and the emotional arc is transformed into emotional interval constraints. Using the set of thematic keywords and the emotional interval constraints as input constraints, a second-level planning is performed on the event sequence in the set of narrative elements to obtain the scene sequence and the narrative function annotation of each scene. The determination of the narrative function annotation is subject to the semantic association constraint of the set of thematic keywords. Based on the narrative function annotation, matching shooting specifications are retrieved from the preset film grammar constraint library. These specifications, along with the narrative intent parameters mapped by the theme keyword set, work together to decompose each scene in the scene sequence into shots, generating shooting parameters and image content descriptions for each shot, thus forming a storyboard sequence.

2. The method according to claim 1, characterized in that, The semantic parsing of the input narrative text yields a set of narrative elements containing event sequences and character relationships, including: The input narrative text is semantically parsed to obtain contextual feature representations; Based on the contextual feature representation, identify the character entities, scene entities, and event sequences in the input narrative text; Based on the event sequence and the role entities, determine the role relationships; The character entity, the scene entity, the event sequence, and the character relationship are defined as the narrative element set.

3. The method according to claim 1, characterized in that, The first-level planning of the narrative element set to generate an emotional arc and narrative theme includes: Identify key events from the event sequence; Based on the key events, the narrative theme is generated; The emotional arc is generated based on the distribution of event types of each event in the event sequence.

4. The method according to claim 3, characterized in that, The step of identifying key events from the event sequence includes: Based on the causal impact range of each event in the event sequence, determine the event impact of each event; Events whose influence exceeds a preset influence threshold are identified as candidate key events; The candidate key events are identified as the key events.

5. The method according to claim 3, characterized in that, The step of generating the emotional arc based on the event type distribution of each event in the event sequence includes: Based on the distribution of event types, the event sequence is divided into four emotional stages: introduction, development, transition, and conclusion. For each of the four emotional stages of introduction, development, transition, and conclusion, an emotional target value is determined; The emotional target values ​​of each emotional stage are arranged in chronological order to obtain the emotional arc.

6. The method according to claim 1, characterized in that, The process of transforming the narrative theme into a set of thematic keywords and the emotional arc into emotional range constraints, based on preset refinement rules, includes: Based on the preset refinement rules, keywords are extracted from the narrative theme to obtain the set of thematic keywords; The emotional target values ​​for each stage of the emotional arc are converted into corresponding numerical ranges. The emotional interval constraints are obtained by combining the numerical intervals in chronological order.

7. The method according to claim 1, characterized in that, The second-level planning of the event sequence in the narrative element set, using the set of thematic keywords and the emotional interval constraints as input constraints, yields the scene sequence and the narrative function annotation of each scene, including: Based on the event boundaries of each event in the event sequence and the set of narrative elements, the event sequence is divided into scenes to obtain the scene sequence. The boundaries of the scene division do not truncate the causal chain of events in the event sequence. Using the emotional interval constraint as an input constraint, an emotional change target is assigned to each scene in the scene sequence; Based on the events contained in each scene and the set of thematic keywords, the narrative function annotation of each scene is determined, so that the narrative function annotation of each scene is semantically associated with at least one thematic keyword in the set of thematic keywords.

8. The method according to claim 1, characterized in that, The step of retrieving matching shooting specifications from a preset film grammar constraint library based on the narrative function annotations, in conjunction with the narrative intent parameters mapped from the theme keyword set, decomposes each scene in the scene sequence into shots, generating shooting parameters and image content descriptions for each shot, thus forming a storyboard sequence, including: Based on the narrative function annotations, the matching shooting specifications are retrieved from the preset film grammar constraint library; Map the set of thematic keywords to narrative intent parameters; Based on the shooting specifications and the narrative intent parameters, each scene in the scene sequence is decomposed into shots to obtain the storyboard sequence, which includes the shooting parameters and content description of each shot.

9. The method according to claim 8, characterized in that, The step of decomposing each scene in the scene sequence into shots based on the shooting specifications and the narrative intent parameters to obtain the storyboard sequence includes: Based on the shooting specifications and the narrative intent parameters, each scene is decomposed into a sequence of shots; For each shot in the shot sequence, determine the shot size, shooting angle, and shot movement as the shooting parameters, wherein the difference in shot size between adjacent shots does not exceed a preset threshold; Generate a description of the scene content for each shot; The shooting parameters and the description of the scene content are organized in scene order to form the storyboard sequence.

10. The method according to claim 9, characterized in that, The step of organizing the shooting parameters and the scene content description into the storyboard sequence according to scene order includes: The shooting parameters of each lens are associated with the description of the scene content to obtain a lens unit; The arrangement order of each shot unit is determined based on the position of each shot in the scene sequence; The shot units are organized according to the aforementioned arrangement order to obtain the shot sequence.

Images