A method and system for generating a shot sequence based on a semantic graph

Abstract

The present application relates to the technical field of artificial intelligence content generation, in particular to a kind of based on semantic atlas's shot sequence generation method and system.The system includes project initialization module, semantic analysis module, shot disassembly module, semantic role atlas module, shot language rule module, cross-modal cue generation module, multi-modal generation module, consistency check and rectification module and shot sequence persistence module.Through the structured semantic analysis of narrative text, shot level disassembly and role semantic atlas construction, the dynamic coupling of role state, shot language and scene semantics is realized.Combined with cross-modal cue generation and multi-modal content generation, the collaborative generation of shot image, dialogue voice and environmental audio is realized.Through directional backwriting rectification and local regeneration mechanism, the visual consistency of the role, the continuity of the shot language and the cross-modal semantic consistency are improved, and the shot generation quality, stability and generation efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence content generation technology, and in particular to a method and system for generating storyboard sequences based on semantic graphs. Background Technology

[0002] With the development of AI-powered content generation technology, the technology of automatically generating film and television storyboards based on scripts, novel texts, or story scripts has gradually attracted attention. Current technologies typically use natural language processing to parse narrative text and combine it with image generation models to generate corresponding scenes, thereby assisting in film and television creation, animation design, game story production, and digital content production.

[0003] However, most existing storyboard generation solutions employ independent processing methods for text parsing and content generation, often generating single frames based solely on local text content. This lack of unified modeling for character states, plot context, and cinematic language rules leads to issues such as inconsistent character appearances, disjointed action transitions, semantic shifts in scenes, and incoherent shot expression between different storyboards. Furthermore, existing solutions have limited cross-modal consistency control over images, dialogue, and ambient sound. When the generated result deviates from the original plot semantics, a complete regeneration is usually required, consuming significant computational resources and making it difficult to guarantee the continuity and stability of the storyboard sequence.

[0004] Therefore, how to provide a semantic graph-based storyboard sequence generation system and method that can improve the visual consistency of characters, the continuity of camera language, and the cross-modal semantic consistency has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies and provide a method and system for generating storyboard sequences based on semantic graphs.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A storyboard sequence generation system based on semantic graphs, comprising:

[0008] The semantic parsing module is used to perform semantic understanding and structured analysis on the input narrative text;

[0009] The storyboard decomposition module is used to establish a shot-level storyboard frame sequence based on the structured semantic data;

[0010] The Semantic Character Graph module is used to build semantic graphs of characters in the plot and continuously record the visual features, actions, emotions, and item holding status of characters as the plot progresses, in order to generate character state parameters.

[0011] The camera language rules module is used to receive the character state parameters, plot rhythm parameters, and scene semantic parameters, and generate camera control parameters for the corresponding storyboard frames based on a dynamic semantic coupling mechanism.

[0012] A cross-modal cue word generation module is used to generate cross-modal cue words for corresponding storyboard frames based on the character state parameters and camera control parameters;

[0013] The multimodal generation module is used to generate image content, dialogue speech content, and ambient sound content of the corresponding storyboard frame based on the cross-modal cue words;

[0014] The consistency verification and correction module is used to perform consistency analysis on the generated results and perform targeted correction when a consistency deviation is detected.

[0015] The camera language rules module and the semantic character graph module establish a character state coupling relationship. The semantic character graph module generates character state parameters based on the character's current emotional state, action intensity, behavior type, and plot conflict level, and outputs them to the camera language rules module to dynamically adjust the camera framing parameters, camera motion parameters, and camera switching frequency.

[0016] The storyboard decomposition module establishes a plot rhythm coupling relationship with the camera language rules module. The storyboard decomposition module generates plot rhythm parameters based on the plot progression speed, dialogue density, action intensity, and emotional change amplitude, and outputs them to the camera language rules module to dynamically adjust the shot duration and camera movement speed.

[0017] The storyboard decomposition module establishes a cross-shot state inheritance relationship with the semantic character graph module. The storyboard decomposition module synchronously outputs the character's spatial position, action posture, clothing state, prop state and emotional state in the previous storyboard frame to the semantic character graph module so as to automatically inherit the corresponding character state based on the temporal continuity between the previous and subsequent shots.

[0018] The consistency verification and correction module establishes a mapping relationship between deviation types and module parameters, and performs directional parameter write-back to the semantic role graph module, shot language rule module and cross-modal prompt word generation module according to the source of deviation, so as to perform local regeneration on the storyboard frames with deviations, thereby forming a closed-loop collaborative control process of semantic parsing, shot control, storyboard generation, consistency verification and dynamic correction.

[0019] Furthermore, the semantic parsing module includes a text preprocessing unit, a narrative structure segmentation unit, a multi-entity recognition unit, an emotion and intent recognition unit, a scene semantic modeling unit, and a semantic structured output unit;

[0020] The text preprocessing unit is used to perform text denoising, format standardization, paragraph normalization, pronoun restoration, and contextual reference resolution on the input narrative text.

[0021] The narrative structure segmentation unit is used to hierarchically decompose the narrative according to chapter level, scene level, event level, and dialogue level;

[0022] The multi-entity recognition unit is used to identify role entities, behavior entities, scene entities, time entities, and environment entities, and to establish semantic relationships between entities;

[0023] The emotion and intention recognition unit is used to identify the character's emotional state and the intention to express the plot.

[0024] The scene semantic modeling unit is used to establish a scene semantic model that includes time parameters, spatial parameters, weather parameters, lighting parameters, and emotional atmosphere parameters.

[0025] Furthermore, the storyboard decomposition module includes a plot rhythm analysis unit, a shot boundary determination unit, a storyboard frame construction unit, a shot continuity control unit, and a storyboard sequence optimization unit;

[0026] The plot rhythm analysis unit is used to generate plot rhythm parameters based on the plot progression speed, dialogue density, action intensity, and emotional change amplitude.

[0027] The shot boundary determination unit is used to determine shot switching nodes based on time changes, scene switching, character action changes, emotional changes, and narrative focus changes.

[0028] The camera continuity control unit is used to perform continuity analysis on the character's spatial position, action posture, clothing status, prop status, and gaze direction in order to maintain logical consistency between consecutive shots.

[0029] Furthermore, the semantic role graph module includes a role node construction unit, a role relationship graph construction unit, a role state management unit, a role reference resource mapping unit, and a role dynamic update unit;

[0030] The character node construction unit is used to create character nodes that include character identity information, visual feature information, action feature information, voice feature information, and emotion mapping information.

[0031] The character status management unit is used to record changes in character clothing, injuries, posture, emotions, and item holding status, and to generate a set of character status parameters.

[0032] The character dynamic update unit is used to automatically update the character status information based on the cross-shot state inheritance relationship and the time continuity between consecutive shot frames, so as to avoid character appearance drift, action breakage or prop loss.

[0033] Furthermore, the lens language rules module includes a lens type decision unit, a camera movement rule matching unit, a composition rule matching unit, a lighting and color rule unit, and an emotion enhancement unit;

[0034] The shot type decision unit is used to determine character shots, scene shots, mixed shots, or transition shots based on character state parameters and plot expression needs.

[0035] The camera movement rule matching unit is used to automatically match push-in, pull-out, panning, tilting, following, or fixed shots based on the intensity of the character's movements and the plot rhythm parameters.

[0036] The composition rule matching unit is used to automatically match central composition, rule of thirds composition, diagonal composition, symmetrical composition or oppressive composition according to the character relationship and scene atmosphere;

[0037] The lighting and color rule unit is used to establish corresponding lighting parameters and color style parameters based on the mood of the scene.

[0038] Furthermore, the camera language rules module establishes a dynamic mapping relationship between character state parameters and camera control parameters. The character state parameters include at least emotional state parameters, action intensity parameters, behavior type parameters, and plot conflict level parameters.

[0039] The lens control parameters include at least lens framing parameters, camera motion parameters, and lens switching frequency parameters;

[0040] Specifically, when a character is detected to be in a state of high conflict and high action intensity, the weight of dynamic camera movement parameters and camera switching frequency parameters is increased; when a character is detected to be in a state of emotional expression or psychological description, the weight of fixed camera parameters, slow push-in camera parameters, and close-up camera parameters is increased.

[0041] Furthermore, the cross-modal cue word generation module includes a character cue word construction unit, a camera cue word construction unit, an environment cue word construction unit, a style cue word construction unit, and a negative constraint unit;

[0042] The character prompt word construction unit is used to generate character appearance description information based on character status parameters;

[0043] The lens cue word construction unit is used to generate photographic language description information based on lens control parameters;

[0044] The negative constraint unit is used to generate negative constraint prompts to suppress character appearance drift, scene offset, composition distortion, and style confusion.

[0045] Furthermore, the consistency verification and correction module includes a role consistency verification unit, a semantic consistency verification unit, an audio-visual consistency verification unit, and an automatic correction unit;

[0046] The automatic correction unit establishes a mapping relationship between deviation types and module parameters.

[0047] When a role consistency deviation is detected, the role visual constraint parameters are written back to the semantic role graph module and the cross-modal cue word generation module.

[0048] When deviations in lens lighting, composition, or emotional expression are detected, the lens control parameters are written back to the lens language rules module.

[0049] When a scene semantic deviation is detected, the scene semantic parameters are written back to the cross-modal prompt word generation module.

[0050] Furthermore, the consistency verification and correction module only performs local regeneration for storyboard frames with consistency deviations, rather than performing overall regeneration for the entire storyboard sequence;

[0051] Furthermore, the storyboard decomposition module synchronously outputs the character's spatial position, action posture, clothing state, prop state, and emotional state in the current storyboard frame to the semantic character graph module to establish cross-shot character state inheritance relationships, thereby improving the visual consistency of characters and the continuity of storyboard sequences.

[0052] Compared with the prior art, the beneficial effects of this application are as follows: The present invention provides a storyboard sequence generation system based on semantic graphs, which establishes a character state coupling relationship between the semantic character graph module and the shot language rule module, and combines it with the plot rhythm coupling mechanism. It can dynamically adjust the shot size, camera movement mode and shot switching frequency according to the character's emotional state, action intensity and plot progression rhythm, thereby improving the shot language coherence and film and television expression ability of the storyboard sequence.

[0053] Meanwhile, by establishing a cross-shot state inheritance relationship between the storyboard decomposition module and the semantic character graph module, this invention can continuously manage the character's spatial position, action posture, clothing state, prop state, and emotional state, thereby effectively reducing the problems of character appearance drift, action breakage, and inconsistent scene expression between different storyboards, and improving the visual consistency of characters and the continuity of the plot.

[0054] Furthermore, this invention establishes a mapping relationship between deviation types and module parameters through a consistency verification and correction module, and performs directional parameter write-back and local regeneration, thereby avoiding the waste of computing resources caused by overall regeneration, improving the efficiency of storyboard generation, and enhancing cross-modal semantic consistency between images, speech, and ambient sound.

[0055] Another technical solution adopted in this application is as follows:

[0056] A semantic graph-based storyboard sequence generation method is applied to a semantic graph-based storyboard sequence generation system. The method includes the following steps:

[0057] S101. Configure the task initialization of the input script, novel text, story script or other narrative text through the project initialization module, and establish task status information and reference resource associations;

[0058] S102. The semantic parsing module performs semantic understanding and structured analysis on the input text to generate structured semantic data including role semantics, scene semantics, emotion semantics and action semantics.

[0059] S103. The storyboard decomposition module performs shot-level decomposition of the plot content based on the structured semantic data, and generates a storyboard frame sequence and corresponding plot rhythm parameters.

[0060] S104. Construct a semantic graph of a character through the semantic character graph module and generate character state parameters;

[0061] S105. The camera control parameters are generated by the camera language rules module based on the character status parameters, plot rhythm parameters and scene semantic parameters, and the camera framing, camera movement mode and camera switching frequency are dynamically adjusted.

[0062] S106. Cross-modal prompt words are generated by fusing character semantic graphs, camera control parameters and scene semantic information through the cross-modal prompt word generation module, and corresponding storyboard frame image content, character dialogue voice content and ambient sound content are generated through the multimodal generation module.

[0063] S107. Perform consistency verification on the generated results through the consistency verification and correction module, and perform parameter write-back correction and local regeneration when a consistency deviation is detected;

[0064] S108. The generated results are stored, restored, and historical versions are managed through the storyboard sequence persistence module to generate storyboard sequence results. Attached Figure Description

[0065] Figure 1 This is a logic block diagram of the present invention.

[0066] Figure 2 This is a block diagram of the initialization module of the present invention.

[0067] Figure 3 This is a block diagram of the semantic parsing module of the present invention.

[0068] Figure 4 This is a block diagram of the storyboard disassembly module of the present invention.

[0069] Figure 5 This is a block diagram of the semantic role graph module of the present invention.

[0070] Figure 6 This is a block diagram of the lens language rules module of the present invention.

[0071] Figure 7 This is a block diagram of the cross-modal prompt word generation module of the present invention.

[0072] Figure 8 This is a block diagram of the multimodal generation module of the present invention.

[0073] Figure 9 This is a block diagram of the consistency verification and correction module of the present invention.

[0074] Figure 10 This is a block diagram of the storyboard sequence persistence module of the present invention. Detailed Implementation

[0075] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments. It should be understood that the described embodiments are only for explaining the present invention and are not intended to limit the scope of protection of the present invention. Other implementation methods obtained by those skilled in the art based on the embodiments of the present invention without creative effort should all fall within the scope of protection of the present invention.

[0076] Reference Figure 1 This embodiment provides a storyboard sequence generation system based on semantic graphs. The system is used to parse scripts, novels, story scripts or other narrative texts and automatically generate storyboard sequence content with character visual consistency, standardization of camera language and cross-modal semantic consistency.

[0077] In this embodiment, the storyboard sequence generation system based on semantic graph mainly includes: a project initialization module 100, a semantic parsing module 200, a storyboard decomposition module 300, a semantic character graph module 400, a shot language rule module 500, a cross-modal cue word generation module 600, a multimodal generation module 700, a consistency verification and correction module 800, and a storyboard sequence persistence module 900. The modules communicate with each other through a unified data bus and achieve full-process collaborative control based on a task state machine.

[0078] To improve the consistency of roles, the continuity of shot language, and the stability of cross-modal semantics during the storyboard generation process, a closed-loop collaborative control relationship is formed between the modules through a dynamic semantic coupling mechanism.

[0079] Specifically, the semantic character graph module 400 outputs character state parameters to the shot language rule module 500 to drive dynamic adjustments to shot size, composition, and camera movement. The shot language rule module 500 synchronously outputs shot control parameters to the cross-modal cue word generation module 600 to establish shot cue words that match the semantics of the plot. The consistency verification and correction module 800 performs deviation analysis on the multimodal generation results and writes the correction results back to the semantic character graph module 400, the shot language rule module 500, and the cross-modal cue word generation module 600 to realize character state correction, shot parameter correction, and cue word weight update, thereby forming a closed-loop control process of semantic parsing, shot generation, consistency verification, and dynamic correction.

[0080] Furthermore, the project initialization module 100 is used to initialize the current storyboard generation task after the user inputs the script, novel text or story script, so as to establish the running foundation for subsequent semantic parsing, storyboard decomposition and cross-modal generation.

[0081] Specifically, refer to Figures 2 to 10 The project initialization module 100 includes a task identifier creation unit 101, a resource loading unit 102, a parameter configuration unit 103, a cache initialization unit 104, a state machine initialization unit 105, and a reference resource pre-association unit 106.

[0082] The task identifier creation unit 101 is used to automatically generate a unique task identifier for the current project after receiving the input text, and to establish a corresponding task directory, data index and task lifecycle record based on the task identifier, so as to realize the association management of subsequent storyboard frames, prompt words, character status and generation results.

[0083] The resource loading unit 102 is used to automatically load preset resources according to the task type, including a character atlas template library, a camera language rule library, a scene semantic template library, a prompt word template library, and a cross-modal generation parameter library.

[0084] Furthermore, the character atlas template library is used to store the appearance parameters, action parameters, and emotion parameters corresponding to different character types; the camera language rule library is used to store the rule parameters corresponding to different camera types, camera movement methods, lighting styles, and composition methods.

[0085] The parameter configuration unit 103 is used to establish the generation parameters of the current project according to user needs or default strategies. The generation parameters include at least storyboard style parameters, screen ratio parameters, shot rhythm parameters, character consistency constraint level, and cross-modal consistency threshold.

[0086] The cache initialization unit 104 is used to establish a character cache area, a camera cache area, a prompt word cache area, and a generation result cache area. The character cache area is used to temporarily record character state change information, the camera cache area is used to record the storyboard breakdown results, the prompt word cache area is used to save the prompt word content generated at different stages, and the generation result cache area is used to cache images, audio, and environmental resources.

[0087] The state machine initialization unit 105 is used to establish the task state machine, which includes an initialization state, a semantic parsing state, a storyboard decomposition state, a prompt word generation state, a multimodal generation state, a consistency verification state, and a task completion state. It is also used to implement breakpoint resumption when the task is abnormally interrupted.

[0088] The reference resource pre-association unit 106 is used to automatically match the corresponding reference resources based on the character name, scene name and style tag in the input text, so as to establish a pre-association relationship between the character reference image, scene reference image and shot style template, thereby improving the efficiency of subsequent prompt word generation and the effect of character consistency control.

[0089] Furthermore, the semantic parsing module 200 is used to perform semantic understanding and structured analysis on the input script, novel text or story script, so as to transform the original unstructured narrative text into structured semantic data that can be used for storyboard generation.

[0090] Specifically, the semantic parsing module 200 includes a text preprocessing unit 201, a narrative structure segmentation unit 202, a multi-entity recognition unit 203, an emotion and intention recognition unit 204, a scene semantic modeling unit 205, and a semantic structured output unit 206.

[0091] The text preprocessing unit 201 is used to perform format standardization processing on the input text, including text denoising, punctuation unification, paragraph normalization, pronoun restoration, and contextual reference resolution, so as to improve the accuracy of subsequent semantic recognition.

[0092] Furthermore, when pronouns such as "he," "she," and "that person" exist in the text, the text preprocessing unit 201 automatically maps them to the corresponding role entities through a context association mechanism to ensure the continuity of role recognition.

[0093] The narrative structure segmentation unit 202 is used to decompose the input text into multi-level narratives and form structured narrative units according to the plot logic.

[0094] Specifically, the narrative structure segmentation unit 202 is hierarchically segmented according to chapter level, scene level, event level, and dialogue level.

[0095] Specifically: chapter-level segmentation is used to identify plot stages; scene-level segmentation is used to identify changes in time, location, and environment; event-level segmentation is used to identify actions and the logic of event progression; and dialogue-level segmentation is used to identify the content of character speech and emotional changes.

[0096] For example, when text markers such as "at the same time," "suddenly," and "the next morning" are detected, the system automatically identifies them as scene transition nodes to trigger the generation of new storyboard segments.

[0097] The multi-entity recognition unit 203 is used to recognize role entities, scene entities, object entities, behavior entities, time entities, and environment entities in text.

[0098] Among them, the character entity is used to represent the character object participating in the plot; the scene entity is used to represent the environment in which the story takes place; the behavior entity is used to represent the character's actions; the time entity is used to represent the change of time; and the environment entity is used to represent the weather, lighting, spatial atmosphere, and background elements.

[0099] Furthermore, the multi-entity recognition unit 203 establishes the relationships between entities to form a preliminary semantic relationship network.

[0100] The emotion and intention recognition unit 204 is used to identify the character's emotional state and the intention to express the plot.

[0101] Specifically, the system determines whether a character is in an emotional state such as sadness, tension, excitement, anger, calmness, or fear through semantic sentiment analysis and contextual reasoning.

[0102] Furthermore, the system identifies implicit directorial intentions in the text, including expressions of oppression, loneliness, emotional climax, and increased conflict.

[0103] For example, when descriptions such as "silently bowing his head," "clenching his fists," and "avoiding eye contact" appear in the text, the system automatically identifies them as emotional reinforcement nodes and increases the weight of the corresponding close-up shots.

[0104] The scene semantic modeling unit 205 is used to build a scene semantic model based on the scene description.

[0105] Specifically, the scene semantic model includes at least: time parameters; spatial parameters; weather parameters; lighting parameters; color atmosphere parameters; and lens atmosphere parameters.

[0106] For example, when the text description is "an abandoned train station in a midnight downpour," the system automatically generates:

[0107] Time parameter: Late at night; Weather parameter: Heavy rain; Environmental parameter: Abandoned train station; Lighting parameter: Low-intensity cold light; Emotional atmosphere parameter: Depressing.

[0108] The semantic structured output unit 206 is used to convert the parsing results into a standardized semantic data structure and output it to the storyboard decomposition module 300.

[0109] Specifically, the output data includes a set of character tags, a set of action tags, a set of scene tags, a set of emotion tags, and a set of camera bias tags.

[0110] Among them, the lens preference label is used to indicate the preferred lens performance mode corresponding to the current semantic scene, including close-up shots, medium shots, panoramic shots, following shots, and dynamic camera movements.

[0111] Through the above semantic parsing process, the automatic conversion from unstructured narrative text to structured film and television semantic data is realized, providing a unified semantic foundation for subsequent storyboard analysis, shot language generation, and cross-modal consistency control.

[0112] Furthermore, the storyboard decomposition module 300 is used to decompose the plot content at the shot level based on the structured semantic data output by the semantic parsing module 200, and generate an executable structured storyboard frame sequence to realize the conversion from narrative text to film and television shot language.

[0113] Specifically, the storyboard decomposition module 300 includes a plot rhythm analysis unit 301, a shot boundary determination unit 302, a storyboard frame construction unit 303, a shot continuity control unit 304, and a storyboard sequence optimization unit 305.

[0114] Among them, the plot rhythm analysis unit 301 is used to identify the rhythm of the current narrative content based on the intensity of the text plot progression, the frequency of character behavior, the magnitude of emotional changes, and the density of dialogue.

[0115] Specifically, when the system identifies high-dynamic scenes such as conflict, chase, and emotional climax, it automatically increases the frequency of camera cuts; when the system identifies low-dynamic scenes such as psychological descriptions, environmental rendering, and emotional exchanges, it automatically reduces the frequency of camera cuts and increases the proportion of close-up shots.

[0116] The lens boundary determination unit 302 is used to determine the lens switching node and form lens boundary information.

[0117] Furthermore, the system automatically determines the boundaries of shots based on changes in time, scene transitions, changes in character actions, sudden changes in emotions, and changes in narrative focus.

[0118] For example, when the text contains semantic markers such as "at the same time," "suddenly," "the next day," or "camera turns," the system automatically determines it as a new shot generation node.

[0119] The storyboard frame construction unit 303 is used to create corresponding shot objects and form structured storyboard frames.

[0120] Specifically, each storyboard frame is assigned a unique shot ID, which includes at least: shot timing parameters; shot size parameters; shot composition parameters; camera motion parameters; character state parameters; scene semantic parameters; emotion parameters; dialogue parameters; and ambient sound parameters.

[0121] Furthermore, the shot parameters include close-up shots, medium shots, long shots, panoramic shots, and overhead shots; the camera movement parameters include push-in shots, pull-out shots, panning shots, tilting shots, tracking shots, and fixed shots.

[0122] The lens continuity control unit 304 is used to ensure logical consistency between the front and rear lenses.

[0123] Specifically, the system avoids camera jumps or narrative breaks by tracking the spatial position of characters, analyzing the continuity of actions, and analyzing the consistency of gaze direction.

[0124] For example, if a character is on the left side of the screen in the previous shot, the system will maintain the consistency of the character's movement direction and line of sight in the next shot.

[0125] The storyboard sequence optimization unit 305 is used to optimize the shot order according to the overall plot structure and automatically adjust the shot duration, shot density and emotional rhythm to improve the cinematic expressiveness of the final storyboard sequence.

[0126] Furthermore, the semantic character graph module 400 is used to construct a character semantic graph to achieve unified management of character visual features, behavioral features, and emotional states, and to improve the consistency of characters in different storyboards.

[0127] Specifically, the semantic role graph module 400 includes a role node construction unit 401, a role relationship graph construction unit 402, a role state management unit 403, a role reference resource mapping unit 404, and a role dynamic update unit 405.

[0128] Among them, the role node construction unit 401 is used to create an independent role node for each role.

[0129] A character node includes at least: character identity information; character visual feature information; character action feature information; character voice feature information; and character emotion mapping information.

[0130] Furthermore, the character's visual features include hairstyle, skin color, body shape, clothing style, age range, and facial feature parameters; the character's action features include habitual actions, behavioral habits, and action intensity preferences; and the character's voice features include speech rate, pitch, tone, and timbre parameters.

[0131] The role relationship diagram building unit 402 is used to establish a network of relationships between roles.

[0132] Specifically, role relationships include intimate relationships, adversarial relationships, cooperative relationships, family relationships, and superior-subordinate relationships.

[0133] Furthermore, the system automatically adjusts the camera style based on the character relationships. For example, when the characters have an intimate relationship, the weight of medium and close-up shots and warm color atmosphere is increased; when the characters have a conflict relationship, the weight of dynamic shots and high-contrast lighting is increased.

[0134] The character status management unit 403 is used to record the character's status changes during the plot progression. Specifically, status changes include changes in clothing, injuries, emotions, age, and environmental adaptation.

[0135] Furthermore, when a character's state changes, the system automatically updates the state information in the character node to prevent character appearance from shifting across different scenes.

[0136] The character reference resource mapping unit 404 is used to establish associations between character nodes and character reference maps, motion reference maps, and style reference maps.

[0137] Specifically, the system determines the weight of the reference resource through a semantic feature matching algorithm and prioritizes calling the reference resource with the highest matching degree.

[0138] The character dynamic update unit 405 is used to automatically correct the character state according to the context during the continuous generation of storyboards and form a dynamic character semantic graph to ensure character continuity.

[0139] Furthermore, the camera language rules module 500 is used to automatically match camera language rules based on the character's state, emotional expression, and scene semantics, and generate standardized camera control strategies.

[0140] Specifically, the lens language rules module 500 includes a lens type decision unit 501, a camera movement rule matching unit 502, a composition rule matching unit 503, a lighting and color rule unit 504, and an emotion enhancement unit 505.

[0141] The shot type decision unit 501 is used to automatically determine the shot type according to the narrative plot and shot expression needs. Shot types include character shots, scene shots, mixed shots and transition shots.

[0142] The camera movement rule matching unit 502 is used to automatically determine the camera movement mode based on the intensity of the action and the pace of the plot.

[0143] For example, when the system recognizes a character running, fighting, or chasing, it automatically calls up a follow shot or a moving shot; when the system recognizes psychological descriptions or static emotional expressions, it prioritizes calling up a fixed shot or a slow-motion shot.

[0144] The composition rule matching unit 503 is used to automatically determine the composition method based on the character relationship and scene atmosphere.

[0145] Specifically, this includes central composition, diagonal composition, rule of thirds composition, symmetrical composition, and oppressive composition.

[0146] The Lighting and Color Rule Unit 504 is used to automatically establish lighting parameters and color style parameters based on the mood of the scene.

[0147] For example, sad scenes use low-saturation cool colors; warm scenes use bright warm colors; and tense scenes use high-contrast lighting.

[0148] The emotion enhancement unit 505 is used to dynamically increase the weight of corresponding shot parameters during plot climaxes, emotional transitions, and conflict phases to enhance the expressiveness of film and television.

[0149] Furthermore, the cross-modal cue word generation module 600 is used to integrate character semantic graphs, camera language rules, reference resources, and scene semantic information, and automatically generate unified cross-modal cue words.

[0150] Furthermore, the character state management unit 403 establishes a state coupling relationship with the camera language rules module 500. Specifically, the character state management unit 403 generates character state parameters based on the character's current emotional state, action intensity, behavior type, and plot conflict level, and outputs them to the camera type decision unit 501 and the camera movement rule matching unit 502 to dynamically adjust the camera framing parameters, camera switching frequency, and camera movement mode.

[0151] For example, when a character is in a state of high tension and is accompanied by running, chasing, or conflict behavior, the system automatically increases the weight of follow shots, dynamic camera movements, and fast-paced camera cuts; when a character is in a calm, reminiscing, or emotional expression state, the system automatically increases the weight of fixed shots, slow push-in shots, and close-up shots, thereby achieving adaptive driving control of camera language based on the character's state.

[0152] The plot rhythm analysis unit 301 establishes a rhythm coupling relationship with the camera movement rule matching unit 502. Specifically, the plot rhythm analysis unit 301 calculates rhythm weight parameters based on the plot progression speed, dialogue density, action intensity, and emotional change amplitude, and synchronizes the rhythm weight parameters to the camera movement rule matching unit 502 to dynamically adjust shot duration, shot switching frequency, and camera movement speed.

[0153] For example, in chase, conflict, and high emotional fluctuation scenes, the system automatically increases the frequency of camera switching and the speed of dynamic camera movement; in environmental rendering or emotional expression scenes, the system automatically extends the duration of shots and reduces the frequency of camera cuts.

[0154] The character dynamic update unit 405 establishes a cross-shot state inheritance relationship with the shot continuity control unit 304. Specifically, the shot continuity control unit 304 synchronously outputs the character's spatial position, action posture, clothing status, prop holding status, and emotional state from the previous shot to the character dynamic update unit 405. The character dynamic update unit 405 automatically inherits the character's state based on the temporal continuity between shots to avoid problems such as character clothing drift, action breaks, missing props, or inconsistent visual characteristics between different shots.

[0155] Specifically, the cross-modal cue word generation module 600 includes a character cue word construction unit 601, a camera cue word construction unit 602, an environment cue word construction unit 603, a style cue word construction unit 604, and a negative constraint unit 605.

[0156] Among them, the character prompt word construction unit 601 is used to automatically generate character appearance descriptions based on the character map;

[0157] The lens cue word construction unit 602 is used to generate photographic language descriptions based on lens rules;

[0158] The environmental cue word construction unit 603 is used to generate weather, location, and lighting descriptions based on scene semantics;

[0159] Style cue word construction unit 604 is used to generate descriptions of art style, color style, and film and television style.

[0160] Furthermore, the negative constraint unit 605 is used to establish negative cue words to avoid character appearance drift, scene shift, camera distortion, and style confusion.

[0161] Specifically, the system automatically identifies and suppresses elements that are inconsistent with the semantics of the current role, thereby improving the stability and consistency of the generation.

[0162] Furthermore, the multimodal generation module 700 is used to generate storyboard images, character dialogue voices, and ambient audio based on cross-modal cue words.

[0163] Specifically, the multimodal generation module 700 includes an image generation unit 701, a speech generation unit 702, an ambient sound generation unit 703, and a time axis synchronization unit 704.

[0164] The image generation unit 701 is used to generate storyboard keyframe images; the speech generation unit 702 is used to generate dialogue audio based on character semantics; and the ambient sound generation unit 703 is used to generate weather sound effects, environmental noise, and background ambient sound effects.

[0165] The timeline synchronization unit 704 is used to synchronize shot time, character dialogue time, and ambient sound playback time to improve the coordination of shot playback.

[0166] Furthermore, the consistency verification and correction module 800 is used to perform multidimensional consistency analysis on the generated results.

[0167] Specifically, the consistency verification and correction module 800 includes a role consistency verification unit 801, a semantic consistency verification unit 802, an audio-visual consistency verification unit 803, and an automatic correction unit 804.

[0168] Among them, the character consistency verification unit 801 is used to compare the consistency of the character's facial features, clothing features and action features;

[0169] The semantic consistency verification unit 802 is used to detect whether the generated image conforms to the semantics of the original text; the audio-visual consistency verification unit 803 is used to detect the degree of matching between the emotion of the image and the emotion of the voice; the automatic correction unit 804 is used to automatically regenerate the prompt words and perform local regeneration when the consistency deviation exceeds the threshold.

[0170] Furthermore, the automatic correction unit 804 is used to establish a mapping relationship between deviation types and module parameters, and to perform directional write-back correction based on the source of the deviation.

[0171] Specifically, when a deviation in character appearance consistency is detected, the automatic correction unit 804 writes back the character visual constraint parameters to the character cue word construction unit 601; when a deviation in lens lighting, composition, or emotional expression is detected, the automatic lens control parameters are written back to the lens cue word construction unit 602 and the lighting and color rule unit 504; when a scene semantic deviation is detected, the automatic scene semantic parameters are written back to the environment cue word construction unit 603 to achieve local cue word reconstruction and local shot regeneration.

[0172] Furthermore, the automatic correction unit 804 performs local regeneration only for the shots corresponding to the deviation area, rather than regenerating the entire segment sequence, in order to reduce computational resource consumption and improve segment generation efficiency.

[0173] Furthermore, the storyboard sequence persistence module 900 is used to realize storyboard result storage, state restoration, and historical version management.

[0174] Specifically, the storyboard sequence persistence module 900 includes a result storage unit 901, a version recording unit 902, a state recovery unit 903, and a task tracking unit 904.

[0175] The result storage unit 901 is used to save storyboard images, audio files, ambient sound files, shot parameters, and character status data; the version recording unit 902 is used to record the results generated at different stages; the status recovery unit 903 is used to restore to the interruption node and continue execution when the task is interrupted; and the task tracking unit 904 is used to record the execution status of each stage and error logs to improve system stability and task traceability.

[0176] This embodiment also provides a storyboard sequence generation method based on semantic graphs, which is applied to the above-mentioned storyboard sequence generation system based on semantic graphs. It is used to perform structured analysis on scripts, novels, story scripts or other narrative texts, and automatically generate storyboard sequence content with character visual consistency, shot language standardization and cross-modal semantic consistency.

[0177] In this embodiment, the storyboard sequence generation method based on semantic graphs includes at least the following steps:

[0178] S101. Receive the input narrative text and perform project initialization processing to establish the runtime environment for the current storyboard generation task.

[0179] Specifically, after receiving the script, novel text, story script, or other narrative text input by the user, the project initialization module 100 automatically creates a corresponding task identifier and establishes a project directory, index data, and lifecycle records corresponding to the task identifier. Simultaneously, the system automatically loads the character atlas template library, camera language rule library, cue word template library, scene semantic template library, and cross-modal generation parameter library according to the current project type. It also configures storyboard style parameters, camera rhythm parameters, character consistency constraint levels, and cross-modal consistency thresholds according to user needs or default strategies.

[0180] Furthermore, the project initialization module 100 establishes a character cache, a shot cache, a cue word cache, and a generated result cache, and initializes the task state machine to enable subsequent semantic parsing, storyboard decomposition, multimodal generation, and consistency correction processes to be executed collaboratively based on a unified state control logic. Simultaneously, the reference resource pre-association unit 106 automatically establishes pre-association relationships between character reference images, scene reference images, and shot style templates based on character names, scene tags, and style descriptions in the input text, thereby improving the efficiency of subsequent storyboard generation and character consistency control capabilities.

[0181] S102. Perform semantic understanding and structured analysis on the input narrative text to establish structured semantic data that can be used for storyboard generation.

[0182] Specifically, the semantic parsing module 200 first performs format normalization, pronoun restoration, contextual reference resolution, and noise cleaning on the input text through the text preprocessing unit 201, and automatically identifies character names, scene descriptions, time nodes, and environmental elements in the text.

[0183] Subsequently, the narrative structure segmentation unit 202 performs hierarchical semantic decomposition according to chapter level, scene level, event level, and dialogue level to form structured narrative units. Among them, when time changes, spatial changes, plot twists, or shifts in narrative focus are detected, the system automatically determines them as new plot segment nodes.

[0184] Furthermore, the multi-entity recognition unit 203 is used to identify character entities, behavior entities, scene entities, time entities, and environmental entities, and to establish semantic relationships between entities to form a preliminary semantic relationship network; the emotion and intention recognition unit 204 is used to identify the character's current emotional state, behavioral intention, and direction of plot expression, and to establish emotion enhancement labels and director intention labels through contextual semantic reasoning; the scene semantic modeling unit 205 establishes a scene semantic model based on time, weather, space, lighting, and atmosphere descriptions.

[0185] Finally, the semantic structured output unit 206 outputs a set of character tags, a set of action tags, a set of scene tags, a set of emotion tags, and a set of shot tendency tags, which serve as the data basis for subsequent storyboard decomposition and shot language generation.

[0186] S103. Based on the structured semantic data, the narrative content is decomposed at the shot level, and a structured storyboard frame sequence is generated.

[0187] Specifically, the storyboard decomposition module 300 first analyzes the current narrative plot's progression speed, action intensity, dialogue density, and emotional fluctuation amplitude through the plot rhythm analysis unit 301 in order to establish plot rhythm weight parameters.

[0188] Subsequently, the shot boundary determination unit 302 determines the shot switching node based on time changes, scene switching, action changes, emotional changes, and changes in narrative focus, and generates shot boundary information.

[0189] Furthermore, the storyboard frame construction unit 303 creates a corresponding storyboard frame object for each shot node and generates a unique shot number. At the same time, it also creates corresponding shot time parameters, shot type parameters, camera motion parameters, character state parameters, environmental parameters, dialogue parameters, and emotion parameters.

[0190] In this embodiment, the plot rhythm analysis unit 301 establishes a rhythm coupling relationship with the camera language rule module 500. When the plot rhythm weight increases, the camera switching frequency and dynamic camera movement weight are automatically increased. When the plot rhythm tends to be stable, the proportion of fixed shots, close-ups and slow-paced shots is automatically increased to achieve dynamic matching between narrative rhythm and film and television camera expression.

[0191] S104. Construct a semantic graph of characters and establish a dynamic management mechanism for character states to improve the consistency of characters across shots.

[0192] Specifically, the semantic role graph module 400 establishes an independent role node for each role and establishes role identity information, visual feature information, action feature information, voice feature information, and emotion mapping information.

[0193] Furthermore, the character relationship graph construction unit 402 establishes a relationship network between characters and dynamically adjusts the camera style strategy based on these relationships. Specifically, when the character relationship is intimate, the weight of warm color atmosphere and medium-close-up shots is increased; when the character relationship is conflicting, the weight of dynamic camera movement and high-contrast lighting is increased.

[0194] At the same time, the character status management unit 403 continuously records changes in character clothing, posture, injury, emotions, and item holding status, and forms a set of character status parameters.

[0195] In this embodiment, the character state management unit 403 establishes a state coupling relationship with the camera language rule module 500. The character's current emotional state, action intensity, behavior type, and plot conflict level are synchronously output to the camera type decision unit 501 and the camera movement rule matching unit 502 to drive the dynamic adjustment of camera framing, camera movement speed, and camera switching frequency.

[0196] S105. Automatically match camera language rules based on character status parameters, plot rhythm parameters, and scene semantic parameters, and generate camera control strategies.

[0197] Specifically, the shot language rules module 500 automatically determines the shot type based on the current character state and narrative expression needs, and establishes camera movement strategy, composition strategy and lighting and color strategy.

[0198] Specifically, when the system identifies a character as being in a state of pursuit, conflict, or high-intensity behavior, the camera movement rule matching unit 502 automatically calls follow shot, moving shot, and fast-paced cut shot strategies; when the system identifies a character as being in a state of psychological description, recollection, or emotional expression, it automatically calls fixed shot, slow push shot, and close-up shot strategies.

[0199] Furthermore, the composition rule matching unit 503 automatically matches central composition, rule of thirds composition, diagonal composition, oppressive composition, or symmetrical composition based on character relationships, the degree of plot conflict, and environmental atmosphere; the lighting and color rule unit 504 establishes corresponding lighting parameters and color style parameters based on the plot mood to form a standardized set of shot control strategies.

[0200] S106: Integrates character semantic graphs, camera language rules, and scene semantic information, and automatically generates cross-modal prompts.

[0201] Specifically, the cross-modal cue word generation module 600 generates character description information, cinematography description information, environment description information, and film and television style description information through the character cue word construction unit 601, shot cue word construction unit 602, environment cue word construction unit 603, and style cue word construction unit 604, and then integrates them according to preset semantic weights.

[0202] Furthermore, the negative constraint unit 605 automatically identifies elements that are inconsistent with the current character state, scene semantics, or camera expression, and generates negative constraint prompts to suppress character drift, scene shift, compositional distortion, and style confusion.

[0203] In this embodiment, the shot language rule module 500 outputs shot control parameters to the shot prompt word construction unit 602 to ensure that the prompt word content is consistent with the shot language expression; the semantic character graph module 400 outputs character state parameters to the character prompt word construction unit 601 to ensure that the character's appearance features, action features and emotional features remain stable in different shots.

[0204] S107. Perform multimodal content generation based on cross-modal cue words to form a storyboard sequence.

[0205] Specifically, the multimodal generation module 700 performs image generation, dialogue speech generation, and environmental audio generation based on cross-modal prompts.

[0206] Among them, the image generation unit 701 is used to generate the corresponding storyboard key frame image; the speech generation unit 702 is used to generate the character dialogue audio according to the character semantics; and the ambient sound generation unit 703 is used to generate weather ambient sound, spatial background sound and emotional atmosphere sound.

[0207] Furthermore, the timeline synchronization unit 704 synchronizes the shot time, character dialogue time, and ambient sound playback time to form a structured storyboard sequence with consistent time correspondence.

[0208] S108. Perform multidimensional consistency analysis on the generated results and determine the type and region of deviation.

[0209] Specifically, the consistency verification and correction module 800 compares and analyzes the character's facial features, clothing features, action posture, and visual continuity through the character consistency verification unit 801; analyzes the degree of matching between the generated image and the original text semantics through the semantic consistency verification unit 802; and analyzes the consistency between voice emotion, camera emotion, and image emotion through the audio-visual consistency verification unit 803.

[0210] When the test results exceed the preset consistency threshold, the system automatically creates a deviation label and determines the source of the deviation, the level of the deviation, and the corresponding deviation area.

[0211] S109. Perform targeted write-back correction and local regeneration according to the deviation type to improve the consistency and generation efficiency of the storyboard sequence.

[0212] Specifically, the automatic correction unit 804 establishes a mapping relationship between deviation types and module parameters, and performs parameter write-back according to the source of the deviation.

[0213] Specifically, when character appearance drift, inconsistent clothing, or broken movements are detected, the character visual constraint parameters are automatically written back to the character cue word construction unit 601 and the character state management unit 403; when abnormal camera lighting, composition shift, or camera expression distortion is detected, the camera control parameters are automatically written back to the camera cue word construction unit 602 and the lighting and color rule unit 504; when scene semantic deviation is detected, the scene semantic parameters are automatically written back to the environment cue word construction unit 603.

[0214] Furthermore, the automatic correction unit 804 performs local regeneration only for the shots corresponding to the deviation area, rather than performing overall regeneration for the entire segment sequence, in order to reduce computational resource consumption and improve segment generation efficiency.

[0215] At the same time, the camera continuity control unit 304 synchronously outputs the character's spatial position, posture information, clothing status, prop status, and emotional status in the current shot to the character dynamic update unit 405 to form a cross-camera character status inheritance mechanism, thereby avoiding problems such as character appearance drift, action breakage, or prop missing in subsequent shots.

[0216] S110: Persistently store the storyboard sequence results and support state recovery and historical version management.

[0217] Specifically, the storyboard sequence persistence module 900 saves storyboard images, dialogue audio, ambient sound files, camera control parameters, and character status parameters through the result storage unit 901; the version recording unit 902 records the generated versions and modification logs at different stages; the status recovery unit 903 is used to restore execution to the interruption point when the task is abnormally interrupted; and the task tracking unit 904 is used to record the execution status of each stage and error logs to improve task stability and process traceability.

[0218] Through the above steps, the automatic conversion from unstructured narrative text to structured film and television storyboard sequences is realized. By using a dynamic semantic coupling mechanism between semantic character maps, shot language rules, and cross-modal cue words, the coordinated control of character visual consistency, shot language continuity, and cross-modal semantic consistency is achieved. At the same time, the quality, stability, and computational efficiency of storyboard generation are improved by combining a directional write-back correction mechanism and a local regeneration mechanism.

[0219] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A storyboard sequence generation system based on semantic graphs, characterized in that, include: The semantic parsing module (200) is used to perform semantic understanding and structured analysis on the input narrative text; The storyboard decomposition module (300) is used to establish a shot-level storyboard frame sequence based on the structured semantic data; The semantic character graph module (400) is used to build a character semantic graph for the characters in the plot and continuously record the visual feature state, action state, emotional state and item holding state of the characters in the process of plot advancement to generate character state parameters. The camera language rules module (500) is used to receive the character state parameters, plot rhythm parameters and scene semantic parameters, and generate camera control parameters for the corresponding storyboard frame based on the dynamic semantic coupling mechanism. A cross-modal cue word generation module (600) is used to generate cross-modal cue words for the corresponding storyboard frame based on the character state parameters and camera control parameters; A multimodal generation module (700) is used to generate image content, dialogue speech content, and ambient sound content of the corresponding storyboard frame based on the cross-modal cue words; The consistency verification and correction module (800) is used to perform consistency analysis on the generated results and perform targeted correction when a consistency deviation is detected. The camera language rules module (500) and the semantic character graph module (400) establish a character state coupling relationship. The semantic character graph module (400) generates character state parameters based on the character's current emotional state, action intensity, behavior type and plot conflict level, and outputs them to the camera language rules module (500) to dynamically adjust the camera shot parameters, camera motion parameters and camera switching frequency. The storyboard decomposition module (300) and the camera language rules module (500) establish a plot rhythm coupling relationship. The storyboard decomposition module (300) generates plot rhythm parameters based on the plot progression speed, dialogue density, action intensity and emotional change range, and outputs them to the camera language rules module (500) to dynamically adjust the shot duration and camera movement speed. The storyboard decomposition module (300) establishes a cross-shot state inheritance relationship with the semantic character graph module (400). The storyboard decomposition module (300) synchronously outputs the character's spatial position, action posture, clothing state, prop state and emotional state in the previous storyboard frame to the semantic character graph module (400) so as to automatically inherit the corresponding character state based on the temporal continuity between the previous and subsequent shots. The consistency verification and correction module (800) establishes a mapping relationship between deviation type and module parameters, and performs directional parameter write-back to the semantic role graph module (400), shot language rule module (500) and cross-modal prompt word generation module (600) according to the source of deviation, so as to perform local regeneration on the storyboard frames with deviation, thereby forming a closed-loop collaborative control process of semantic parsing, shot control, storyboard generation, consistency verification and dynamic correction.

2. The storyboard sequence generation system based on semantic graphs according to claim 1, characterized in that, The semantic parsing module (200) includes a text preprocessing unit (201), a narrative structure segmentation unit (202), a multi-entity recognition unit (203), an emotion and intention recognition unit (204), a scene semantic modeling unit (205), and a semantic structured output unit (206). The text preprocessing unit (201) is used to perform text denoising, format standardization, paragraph normalization, pronoun restoration and contextual reference resolution on the input narrative text. The narrative structure segmentation unit (202) is used to perform hierarchical narrative decomposition according to chapter level, scene level, event level and dialogue level; The multi-entity recognition unit (203) is used to recognize role entities, behavior entities, scene entities, time entities and environment entities, and to establish semantic association relationships between entities; The emotion and intention recognition unit (204) is used to recognize the character's emotional state and the intention to express the plot; The scene semantic modeling unit (205) is used to establish a scene semantic model including time parameters, spatial parameters, weather parameters, lighting parameters and emotional atmosphere parameters.

3. The storyboard sequence generation system based on semantic graphs according to claim 1, characterized in that, The storyboard decomposition module (300) includes a plot rhythm analysis unit (301), a shot boundary determination unit (302), a storyboard frame construction unit (303), a shot continuity control unit (304), and a storyboard sequence optimization unit (305). The plot rhythm analysis unit (301) is used to generate plot rhythm parameters based on the plot progression speed, dialogue density, action intensity and emotional change amplitude. The camera boundary determination unit (302) is used to determine camera switching nodes based on time changes, scene switching, character action changes, emotional changes, and narrative focus changes. The camera continuity control unit (304) is used to perform continuity analysis on the character's spatial position, action posture, clothing status, prop status, and gaze direction in order to maintain logical consistency between the preceding and following shots.

4. The storyboard sequence generation system based on semantic graphs according to claim 1, characterized in that, The semantic role graph module (400) includes a role node construction unit (401), a role relationship graph construction unit (402), a role state management unit (403), a role reference resource mapping unit (404), and a role dynamic update unit (405). The character node construction unit (401) is used to create character nodes that include character identity information, visual feature information, action feature information, voice feature information and emotion mapping information. The character status management unit (403) is used to record changes in character clothing, injuries, posture, emotions, and item holding status, and to generate a set of character status parameters. The character dynamic update unit (405) is used to automatically update the character status information based on the cross-shot state inheritance relationship and the time continuity between the previous and next shot frames, so as to avoid character appearance drift, action breakage or prop loss.

5. A storyboard sequence generation system based on semantic graphs according to claim 1, characterized in that, The lens language rules module (500) includes a lens type decision unit (501), a camera movement rule matching unit (502), a composition rule matching unit (503), a lighting and color rule unit (504), and an emotion enhancement unit (505). The shot type decision unit (501) is used to determine the character shot, scene shot, mixed shot or transition shot according to the character state parameters and the needs of the plot expression. The camera movement rule matching unit (502) is used to automatically match push-in camera, pull-out camera, panning camera, tilting camera, following camera or fixed camera according to the intensity of the character's actions and the plot rhythm parameters; The composition rule matching unit (503) is used to automatically match the central composition, rule of thirds composition, diagonal composition, symmetrical composition or oppressive composition according to the character relationship and scene atmosphere; The lighting and color rule unit (504) is used to establish corresponding lighting parameters and color style parameters according to the mood of the scene.

6. The storyboard sequence generation system based on semantic graphs according to claim 1, characterized in that, The camera language rules module (500) establishes a dynamic mapping relationship between character state parameters and camera control parameters. The character state parameters include at least emotional state parameters, action intensity parameters, behavior type parameters, and plot conflict level parameters. The lens control parameters include at least lens framing parameters, camera motion parameters, and lens switching frequency parameters; Specifically, when a character is detected to be in a state of high conflict and high action intensity, the weight of dynamic camera movement parameters and camera switching frequency parameters is increased; when a character is detected to be in a state of emotional expression or psychological description, the weight of fixed camera parameters, slow push-in camera parameters, and close-up camera parameters is increased.

7. The storyboard sequence generation system based on semantic graphs according to claim 1, characterized in that, The cross-modal cue word generation module (600) includes a character cue word construction unit (601), a camera cue word construction unit (602), an environment cue word construction unit (603), a style cue word construction unit (604), and a negative constraint unit (605). The character prompt word construction unit (601) is used to generate character appearance description information based on character status parameters; The lens cue word construction unit (602) is used to generate photographic language description information based on lens control parameters; The negative constraint unit (605) is used to generate negative constraint prompts to suppress character appearance drift, scene offset, composition distortion and style confusion.

8. A storyboard sequence generation system based on semantic graphs according to claim 1, characterized in that, The consistency verification and correction module (800) includes a role consistency verification unit (801), a semantic consistency verification unit (802), an audio-visual consistency verification unit (803), and an automatic correction unit (804). The automatic correction unit (804) establishes a mapping relationship between deviation types and module parameters; When a role consistency deviation is detected, the role visual constraint parameters are written back to the semantic role graph module (400) and the cross-modal cue word generation module (600); When a deviation in lens lighting, composition, or emotional expression is detected, the lens control parameters are written back to the lens language rules module (500); When a scene semantic deviation is detected, the scene semantic parameters are written back to the cross-modal prompt word generation module (600).

9. A storyboard sequence generation system based on semantic graphs according to claim 1, characterized in that, The consistency verification and correction module (800) only performs local regeneration for the storyboard frames with consistency deviations, rather than performing overall regeneration for the entire storyboard sequence; Furthermore, the storyboard decomposition module (300) synchronously outputs the character's spatial position, action posture, clothing state, prop state, and emotional state in the current storyboard frame to the semantic character graph module (400) to establish a cross-shot character state inheritance relationship, thereby improving the visual consistency of the characters and the continuity of the storyboard sequence.

10. A method for generating storyboard sequences based on semantic graphs, characterized in that, The method, applied to a semantic graph-based storyboard sequence generation system, includes the following steps: S101. The project initialization module (100) performs task initialization configuration on the input script, novel text, story script or other narrative text, and establishes task status information and reference resource associations. S102. The semantic parsing module (200) performs semantic understanding and structured analysis on the input text to generate structured semantic data including role semantics, scene semantics, emotion semantics and action semantics. S103. The storyboard decomposition module (300) performs shot-level decomposition of the plot content based on the structured semantic data and generates a storyboard frame sequence and corresponding plot rhythm parameters. S104. Construct a semantic graph of a character through the semantic character graph module (400) and generate character state parameters; S105. The camera control parameters are generated by the camera language rules module (500) based on the character status parameters, plot rhythm parameters and scene semantic parameters, and the camera framing, camera movement mode and camera switching frequency are dynamically adjusted. S106. The cross-modal prompt word generation module (600) integrates the character semantic map, camera control parameters and scene semantic information to generate cross-modal prompt words, and generates the corresponding storyboard frame image content, character dialogue voice content and ambient sound content through the multimodal generation module (700). S107. The consistency verification and correction module (800) performs consistency verification on the generated results, and performs parameter write-back correction and local regeneration when a consistency deviation is detected. S108. The generated results are stored, restored and managed in terms of state through the storyboard sequence persistence module (900) to generate storyboard sequence results.

Images