A method and system for automatically generating a shot breakdown of an episode based on lens semantic decomposition

Abstract

The application discloses a kind of based on shot semantic decomposition's episode shot automatic generation method and system, it is related to digital content generation technical field, the application is realized from script text to shot structure hierarchical semantic decomposition by based on emotional curve's semantic narration unit division mechanism, make shot boundary and plot tension change match, significantly improve shot rhythm and narrative coherence;Meanwhile, by structuring coding to foreground subject, background environment and its spatial relationship, and combining object spatial relationship triple and foreground-background position calculation rule, realize the automatic composition of shot picture and semantic consistent fusion, avoid the picture logic mismatch problem caused by only relying on keyword matching in the prior art.

Description

Technical Field

[0001] This invention relates to the field of digital content generation technology, and in particular to a method and system for automatically generating storyboards for TV series based on shot semantic decomposition. Background Technology

[0002] With the continuous improvement of the film and television industry and the significant increase in the demands of streaming media platforms for content production efficiency, the production process of TV series is gradually evolving from the traditional manual division of labor to a digital and intelligent model. In the early creative stage, storyboard design, as a crucial intermediary link connecting the literary script and the audiovisual presentation, undertakes key functions such as visualization of narrative structure, planning of shot language, and pre-visualization of scene scheduling. In recent years, the development of natural language processing, computer vision, and multimodal generation technologies has made the automatic generation of images or video clips based on text content a research hotspot, with some systems capable of generating concept images or sketch-level shots based on the script text. However, existing technologies mostly focus on single-time mapping from text to image or template-based shot splicing, lacking systematic modeling of the deep correspondence between the internal narrative structure of the script and the semantics of the shots, especially in the division of continuous semantic units, the structured expression of shot parameters, and the generation of shot movement logic, which are still in their infancy. In addition, existing automated tools often only mechanically segment scenes based on scene segmentation symbols, ignoring the impact of the plot's emotional changes on the rhythm of the shots and the center of gravity of the composition, making it difficult to generate storyboard sequences with narrative coherence.

[0003] Furthermore, current solutions for automatic storyboard generation also have significant limitations in image material retrieval and scene fusion. On the one hand, most systems extract single images from the material library based on keyword matching, without structurally decomposing and combining foreground, background, and scene layers, resulting in generated images lacking spatial hierarchy and depth logic. On the other hand, in terms of shot duration allocation, motion trajectory setting, and transition processing between storyboards, fixed rules or simple linear interpolation methods are mostly used, lacking dynamic adjustment mechanisms for changes in plot tension and visual rhythm control. Therefore, in practical applications, a large amount of manual intervention is still required, which restricts creative efficiency and large-scale production capabilities. Especially in the production of continuous content at the series level, how to achieve automated batch generation of storyboards while ensuring narrative consistency has become a major bottleneck restricting the implementation of intelligent film and television production systems. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this section, the abstract and title of the invention. Such simplifications or omissions shall not be used to limit the scope of the present invention.

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by this invention is that existing automatic storyboard generation technologies suffer from insufficient semantic decomposition depth and incomplete expression of shot parameters.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: an automatic generation method for TV series storyboards based on shot semantic decomposition, characterized by comprising: receiving a script; dividing the script into several consecutive semantic narrative units based on scene transition markers and plot emotion curves in the script; for each semantic narrative unit, parsing the contained text descriptions to generate a corresponding storyboard sequence, the storyboard sequence including shot parameters and image composition descriptions; retrieving foreground and background images from a preset image library according to the storyboard sequence and merging them into storyboard images; synthesizing the storyboard image sequence into a video stream according to the duration and shot motion parameters of each storyboard, performing transition processing, and outputting a TV series storyboard video file.

[0008] As a preferred embodiment of the present invention, the division of the semantic narrative unit includes: parsing the script, extracting scene switching markers from the script, dividing the script into multiple scene units based on the scene switching markers, with each scene unit corresponding to a continuous plot; for each scene unit, extracting the text content therein, and counting the number of positive and negative emotional words in a preset emotional word library for each sentence; using the difference between the number of positive and negative emotional words as the emotional intensity value of the corresponding text content, and generating the emotional intensity sequence of the scene unit; if there are no emotional words, the emotional intensity value is set to 0.

[0009] As a preferred embodiment of the present invention, the division of the semantic narrative unit further includes: Identify local minima in the emotional intensity sequence; using the local minima as the segmentation boundary, further segment each scene unit into at least one semantic narrative unit, each semantic narrative unit containing the complete emotional change range from the peak to the trough of emotional intensity; if there are no local minima in the emotional intensity sequence, then the scene unit as a whole is treated as a semantic narrative unit.

[0010] As a preferred embodiment of the present invention, the step of parsing the included text description and generating the corresponding storyboard sequence includes: receiving the semantic narrative unit, splitting the text into several clauses according to sentences, with each clause corresponding to a potential storyboard; for each clause, extracting nouns as subjects as core foreground candidates, extracting verb phrases as predicates or accompanying adverbs as action descriptions, and extracting nouns as objects or prepositional objects as background object candidates; for each clause, determining the core foreground candidate as the core foreground subject, and simultaneously detecting whether the core foreground candidate is modified by a preset emphatic modifier; if modified, marking the shot type in the shot parameters of the storyboard as a close-up; otherwise, the shot type defaults to medium. Scenery; other nouns not selected as core foreground are categorized into background environment information; for the core foreground subject of each clause, the verb corresponding to the core foreground subject is extracted from the action description as the action type, the adverb modifying the verb is extracted as the action modifier, and the modifier describing the subject's posture is also extracted. The above information is combined to form foreground subject encoding data; for each object noun in the background environment information of each clause, prepositional phrases describing the spatial relationship between objects are extracted from the text, and object spatial relationship triples are generated based on the prepositional phrases. The triples include the first object, the spatial relationship word, and the second object. The object nouns and the object spatial relationship triples are summarized into background environment encoding data.

[0011] As a preferred embodiment of the present invention, the step of parsing the text description and generating the corresponding storyboard sequence package further includes: extracting prepositional phrases describing the spatial relationship between the core foreground subject and the background environment in each clause, generating foreground-background spatial relationship data, including one or more spatial relationship triples; if there is no explicit spatial relationship description in the text, it is assumed that the foreground subject is located in the center of the frame and has no specific background association; extracting camera movement descriptions from each clause as camera movement parameters, if none are found, it is assumed to be stationary; simultaneously, determining composition position parameters based on the composition descriptions appearing in the text, if none are found, setting a default composition based on the shot size; combining the foreground subject encoding data, background environment encoding data, foreground-background spatial relationship data, and the determined camera parameters corresponding to each clause to form the storyboard composition data corresponding to the current clause; arranging the storyboard composition data corresponding to all clauses within the same semantic narrative unit in clause order, and outputting the storyboard sequence of the semantic narrative unit.

[0012] As a preferred embodiment of the present invention, the fusion into a storyboard includes: retrieving a matching image from a preset character image library based on the foreground subject encoding data; if multiple angles or pose variations exist, selecting the image that best matches the description; if no perfect match exists, using a default pose image and labeling it with a mask; outputting the foreground image and its corresponding binary mask; retrieving an image of each object from a preset scene object library based on object names and spatial relationship triples in the background environment encoding data, and combining them on a blank canvas according to the spatial relationship triples to generate a background image layer; recording the bounding box coordinates of each object on the background canvas during combination to form a background object position mapping table.

[0013] As a preferred embodiment of the present invention, the fusion into a storyboard further includes: determining the initial reference point coordinates of the foreground subject based on the composition position; for each spatial relationship triplet in the foreground-background spatial relationship data, obtaining the bounding box coordinates of the corresponding background object from the background object position mapping table; calculating the coordinates of the candidate center point of the foreground subject that satisfies the spatial relationship according to the preset relative position rules corresponding to the relation words; if there are multiple spatial relationship triplets, then performing a weighted average on all the obtained candidate center point coordinates, or selecting one according to a preset priority, as the final center point coordinates of the foreground subject; if there is no foreground-background spatial relationship data, then using the initial reference point as the center point of the foreground subject, calculating the placement coordinates of the upper left corner of the foreground image on the background canvas; covering the background image layer with the calculated placement coordinates, and using a foreground mask for pixel blending: replacing background pixels with foreground pixels in the mask area, retaining the background in the remaining areas, and outputting the fused storyboard.

[0014] As a preferred embodiment of the present invention, the step of synthesizing the sequence of storyboard images into a video stream includes: receiving the fused image of each storyboard and the duration and camera motion parameters in the corresponding shot parameters; setting a basic continuous display time for each image, but if the camera motion parameters are not static, a dynamic image needs to be generated within the basic continuous display time; for each storyboard, generating a dynamic video segment based on the camera motion parameters; aggregating the video segments generated by all storyboards within the same semantic narrative unit, arranging them in sentence order to form a video segment sequence of the semantic narrative unit; and splicing all the video segments after transition processing in chronological order, encoding them into a video stream, and outputting it as a series storyboard video file.

[0015] As a preferred embodiment of the present invention, the step of generating dynamic video segments based on camera motion parameters includes: if the camera motion is static, the fused image is directly used as the static frame of the segment duration; if the camera motion is zooming in or out, the fused image is continuously linearly scaled within the segment duration according to a preset scaling factor or a scaling factor extracted from the text to generate a scaled dynamic segment; if the camera motion is panning, the fused image is continuously linearly translated within the segment duration according to a preset translation distance or the position of a target reference extracted from the text to generate a panning dynamic segment; the starting point of the translation is the current composition position, and the ending point is the corresponding position determined according to the preset translation distance, or the expected position of the target reference in the image; if the camera motion is of other types, a corresponding affine transformation dynamic segment is generated according to a transformation path determined by preset transformation parameters or text semantics; all dynamic segments are generated at a fixed frame rate.

[0016] On the other hand, the present invention also provides an automatic storyboard generation system for TV series based on shot semantic decomposition, comprising: a script unit division module, which receives the script and divides the script into several continuous semantic narrative units based on scene switching markers and plot emotion curves in the script; a storyboard generation module, which parses the text description contained in each semantic narrative unit and generates a corresponding storyboard sequence, wherein the storyboard sequence includes shot parameters and a description of the scene composition; an image retrieval and fusion module, which retrieves foreground and background images from a preset image library according to the storyboard sequence and fuses them into storyboard images; and a video synthesis and output module, which synthesizes the storyboard image sequence into a video stream according to the duration and shot motion parameters of each storyboard, performs transition processing, and outputs a TV series storyboard video file.

[0017] The beneficial effects of this invention are as follows: This invention achieves hierarchical semantic decomposition from script text to shot structure through a semantic narrative unit division mechanism based on emotion curves, making the shot boundaries match the changes in plot tension and significantly improving the shot rhythm and narrative coherence; at the same time, by structurally encoding the foreground subject, background environment and their spatial relationships, and combining the object spatial relationship triplet and foreground-background position calculation rules, it achieves automatic composition and semantic consistency integration of shot images, avoiding the problem of image logic mismatch caused by relying solely on keyword matching in the prior art.

[0018] Furthermore, this invention embeds camera motion parameters and duration parameters into the video synthesis process, supporting dynamic scaling, translation, and affine transformation generation based on text semantics. This enables the output storyboard video to have continuous motion effects and expressive capabilities of camera language, thereby improving the overall intelligence and industrial adaptability of automatic storyboard generation for TV series. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart illustrating an automatic generation method for TV series storyboards based on shot semantic decomposition, as shown in this invention.

[0020] Figure 2 This is a structural diagram of an automatic storyboard generation system for TV series based on shot semantic decomposition, as shown in this invention. Detailed Implementation

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0022] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort should fall within the scope of protection of this invention.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] According to an embodiment of the present invention, in combination Figure 1 The flowchart shown illustrates an automatic storyboard generation method for TV series based on shot semantic decomposition, comprising: S1: Receive the script and, based on the scene transition markers and emotional curves in the script, divide the script into several continuous semantic narrative units.

[0025] S1.1: Analyze the script, extract scene transition markers from the script, and divide the script into multiple scene units based on the scene transition markers. Each scene unit corresponds to a continuous plot.

[0026] The script is a formatted text file or structured data stream containing elements such as scene titles, dialogue, and action descriptions. The script text is traversed, and scene transition markers are identified through regular expression matching or a predefined tag library. These scene transition markers include, but are not limited to, scene titles in standard script formats (e.g., interior-study-daytime), explicit scene separators (e.g., cut to, fade in), or scene codes embedded in script editing software.

[0027] Based on the identified scene transition markers, the script is divided into N consecutive text blocks, each defined as a scene unit. Each scene unit is continuous in time and space, corresponding to a complete dramatic scene in the script. The output is a sequence of scene units, each scene unit... It includes all text descriptions, dialogues, and action instructions within the scene.

[0028] S1.2: For each scene unit, extract the text content and count the number of positive and negative sentiment words in the preset sentiment word library for each sentence.

[0029] In specific operations, for each scene unit output in step S1.1 The text content is segmented into natural sentences to obtain a set of sentences. Load the preset emotional vocabulary, which includes a set of positive emotional words (such as joy, brilliance, warmth) and a set of negative emotional words (such as sadness, gloom, despair).

[0030] The sentiment lexicon can be constructed through manual annotation or pre-training based on sentiment analysis of large-scale corpora.

[0031] For each sentence Perform the following operations: segment and tag the sentence with words, extracting adjectives, verbs, nouns, and other content words; iterate through the extracted content words and count the number of words that match the set of positive sentiment words. And the number of words in the set of negative sentiment words. Record the statistical pairs for each sentence. .

[0032] S1.3: The difference between the number of positive sentiment words and the number of negative sentiment words is used as the sentiment intensity value of the corresponding text content to generate the sentiment intensity sequence of the scene unit; if there are no sentiment words, the sentiment intensity value is set to 0.

[0033] Specifically, emotional intensity value The expression is: in, This is a preset scaling factor used to map the emotion value to a suitable numerical range (e.g., [-10, 10] or [0, 100]).

[0034] If a sentence does not match any sentiment word (i.e.) and ),but Set it directly to 0.

[0035] Furthermore, regarding scene units The emotional intensity value of all sentences within the scene is calculated sequentially, and the sentences are arranged in their original order in the script to generate the emotional intensity sequence for that scene unit. Each This corresponds to the emotional intensity coordinate of a sentence within the narrative flow.

[0036] It should be noted that this invention transforms discrete emotional word statistics into continuous emotional intensity curves, which intuitively reflects the fluctuations in the emotional dimension of the plot. Furthermore, the use of normalization calculation eliminates the influence of sentence length on emotional values, making scene units of different lengths comparable.

[0037] S1.4: Identify local minima in the emotional intensity sequence.

[0038] The local minimum point is defined as: for a point in the sequence... If satisfied (That is, the value of this point is not greater than the values ​​of its left and right adjacent points), then it is determined that... This is a local minimum point.

[0039] For the start and end points of the sequence, they are processed according to preset rules: if the start point of the sequence is... Less than or equal to Then Mark as candidate boundary; if sequence end Less than or equal to Then Mark as candidate boundaries.

[0040] Furthermore, if multiple consecutive points satisfy the minimum condition, only the middle point or the first point is taken as the representative.

[0041] S1.5: Using the local minimum point as the segmentation boundary, each scene unit is further divided into at least one semantic narrative unit, and each semantic narrative unit contains a complete emotional change range from the peak value to the valley value of the emotional intensity.

[0042] The internal structure of each semantic narrative unit is as follows: it starts from the peak of emotional intensity, undergoes a decline, and ends at the minimum point; or it starts from the minimum point, undergoes an increase, and ends at the peak. In other words, each unit contains a complete arc of emotional change. This ensures that each semantic narrative unit has inherent emotional integrity, and the subsequent storyboards generated for each unit will maintain a consistent visual emotional tone.

[0043] This segmentation based on emotional changes is more narratively plausible than segmentation based solely on text length, and the resulting video storyboards are more in line with the audience's viewing experience.

[0044] Furthermore, if the emotional intensity sequence has no local minimum points (i.e., monotonous or flat), then the scene unit as a whole is regarded as a semantic narrative unit.

[0045] The criterion for determining whether a sequence is monotonic or flat is: for all points in the sequence, there is no point that simultaneously satisfies the condition of being less than or equal to its left and right adjacent points. Sequences that are completely flat (where all E values ​​are equal) also fall into this category.

[0046] S1.6: Gather all semantic narrative units obtained from scene unit segmentation, arrange them in script order, and output a sequence of semantic narrative units. Each semantic narrative unit contains a continuous text content and has a clear boundary of emotional change.

[0047] As can be seen, this invention combines physical scene segmentation with emotional semantic segmentation to generate a composite processing unit that conforms to the script structure and reflects the emotional rhythm, providing a precise and semantically rich processing granularity for subsequent automated storyboard generation.

[0048] S2: For each semantic narrative unit, parse the contained text description and generate a corresponding storyboard sequence, which includes shot parameters and a description of the scene composition.

[0049] S2.1: Receive the semantic narrative unit, split the text into several clauses according to sentences, and each clause corresponds to a potential scene; for each clause, extract the noun as the subject as the core foreground candidate, extract the verb phrase as the predicate or accompanying adverbial as the action description, and extract the noun as the object or prepositional object as the background object candidate.

[0050] The text is divided into several clauses through natural language preprocessing, including clause segmentation, word segmentation, part-of-speech tagging, and dependency parsing. Each clause corresponds to a potential scene. Sentence boundaries are determined by punctuation marks such as periods, question marks, and exclamation marks, or by semantic integrity.

[0051] For each clause, the following components are extracted using dependency parsing tools: Identify noun phrases that function as subjects, typically the core agent or theme of the sentence, as core foreground candidates. If the sentence is in the passive voice, extract the logical subject or determine it based on semantic role labeling; Extract the verb phrase that serves as the core of the predicate, including the verb itself and its accompanying adverbial (such as "slowly" in "walk slowly") as an action description; Extract noun phrases that function as objects or objects of prepositions, including direct objects, indirect objects, and objects in prepositional phrases (such as the table in "on the table"), as candidates for background objects.

[0052] It should be noted that clause-level splitting ensures that each storyboard corresponds to a complete narrative micro-unit, avoiding visual confusion caused by long sentences containing multiple actions, and improving the semantic alignment accuracy between storyboards and text.

[0053] S2.2: For each clause, the core foreground candidate is determined as the core foreground subject (the default subject is the foreground), and at the same time, it is detected whether the core foreground candidate is modified by a preset emphasis modifier.

[0054] Among them, the emphatic modifiers include close-up, focus, or emphasis. If modified, the shot type in the lens parameters of the scene is marked as close-up; otherwise, the shot type is medium shot by default.

[0055] Other nouns that were not selected as core foreground elements (including original background object candidates) are included in the background environment information.

[0056] S2.3: For the core foreground subject of each clause, extract the verb corresponding to the core foreground subject from the action description as the action type, extract the adverb that modifies the verb as the action modifier, and extract the modifier describing the subject's posture (such as facing the camera, turning away from the camera, etc.). Combine the above information to form foreground subject encoding data (including subject name, action type, action modifier, and posture).

[0057] If a component is missing, it is filled with an empty value or a default value (such as no action or default posture).

[0058] S2.4: For each object noun in the background environment information of each clause, extract the prepositional phrases in the text that describe the spatial relationship between objects, and use dependency syntax to generate object spatial relationship triples based on the prepositional phrases.

[0059] The triples include the first object, the spatial relation term, and the second object. The object nouns and the object spatial relation triples are summarized into background environment coding data.

[0060] Spatial relation words include prepositions or phrases of location such as up, down, beside, and behind.

[0061] All object names and their associated triples are aggregated to form the contextual encoding data. For example, for the sentence "The cat is under the table and next to a book," the triples (cat, under, table) and (book, next to, cat) can be generated. If there is no explicit description of the spatial relationship between objects, only the object names are recorded, defaulting to random placement or a regular layout.

[0062] It should be noted that this invention structures the spatial layout information in the text, enabling background synthesis to precisely place each element according to the relation triples, ensuring that the generated scene is consistent with the original description. This explicit relation representation avoids the spatial misalignment problems that may occur with direct generation.

[0063] S2.5: A relation extraction method based on dependency parsing can be used to extract prepositional phrases (such as "next to" or "towards") that describe the spatial relationship between the core foreground subject and the background environment in each clause, generating foreground-background spatial relationship data, which includes one or more spatial relationship triples (foreground subject, relational word, background object).

[0064] If the text does not explicitly describe spatial relationships, it will be assumed that the foreground subject is located in the center of the image and has no specific background association. During subsequent image blending, the composition position parameters will be used for further processing.

[0065] It should be noted here that by explicitly capturing the relative positions of the foreground and background, the positioning of the subject in the scene has a textual basis, avoiding visual errors such as the subject floating or being misaligned. This data, together with the composition position parameters, achieves a precise conversion from literary description to visual composition.

[0066] S2.6: Extract camera motion descriptions (such as zoom in, zoom out, pan, etc.) or natural language variations (such as slow camera movement) from each clause as camera motion parameters. If none are found, the default is static. Extraction methods can include keyword matching or semantic role recognition based on dependency syntax. If no motion description is found, the camera motion parameters default to static.

[0067] Simultaneously, the composition position parameters are determined based on the composition descriptions appearing in the text (such as left side of the screen, center, etc.); if none are found, the default composition is set according to the following rules: if the shot is a close-up, the default composition position is center; if the shot is a medium shot, the default composition position is the golden ratio point (i.e., the subject is located at approximately 1 / 3 of the horizontal direction of the screen). In addition, the duration parameter of the storyboard is initially set as the estimated duration of the sentence text reading (which can be calculated by multiplying the number of words in the text by the average speech rate coefficient) or a preset fixed duration (such as 2 seconds).

[0068] S2.7: Combine and encapsulate the foreground subject encoding data (including pose), background environment encoding data, foreground-background spatial relationship data, and determined shot parameters (shot size, shot movement, composition position, duration) corresponding to each clause to form the storyboard composition data corresponding to the current clause.

[0069] S2.8: Arrange the storyboard data corresponding to all clauses within the same semantic narrative unit according to the original order of the clauses in the script text content contained in the semantic narrative unit, and output the storyboard sequence of the semantic narrative unit.

[0070] S3: Based on the storyboard sequence, retrieve foreground and background images from the preset image library and merge them into a storyboard frame.

[0071] S3.1: Based on the foreground subject encoding data, retrieve matching images from the preset character image library. If there are multiple angles or pose variations, select the image that best matches the description; if there is no perfect match, select the default pose image and label it with a mask (foreground outline area, background transparent); output the foreground image and the corresponding binary mask.

[0072] The character image library stores image resources in a hierarchical structure, with each character corresponding to multiple image variants. The variant index dimensions include action type, pose, viewpoint (front / side, etc.), and expression. Image files are stored in RGBA format, where the alpha channel records transparency information, or additionally stores a binary mask image (white areas represent foreground outlines).

[0073] The retrieval process employs a multi-attribute matching algorithm: First, the character is located using the subject name as the primary key. Then, the matching degree with (action type, action modifier, pose) is calculated in the character's variant set. The matching degree calculation can be based on exact tag matching or semantic similarity (if the tags use synonym expansion). The image with the highest matching degree is selected as the foreground image. If the matching degree is lower than a preset matching threshold (which can be set according to actual needs) or no exact match exists, the default pose image of the character (usually standing and facing forward) is selected, and the corresponding binary mask is output. If the subject is not found in the character database, a null value is returned, and exception handling is triggered (such as skipping the scene or using a placeholder image).

[0074] The output data packet includes the foreground image layer (RGBA image with an alpha channel) and the corresponding binary mask (single channel, pixel value 0 or 255).

[0075] S3.2: Based on the object names and spatial relationship triples in the background environment encoding data, retrieve the image of each object from the preset scene object library, and combine them on the blank canvas according to the spatial relationship triples to generate a background image layer.

[0076] The scene object library stores images of various objects (such as furniture, plants, buildings, etc.). Each object image also comes with an alpha channel or a binary mask, and some complex objects have pre-defined key point annotations (such as the center of a table, the bottom of a tree canopy, etc.).

[0077] Ideally, the background generation process is divided into a combination mode and an overall retrieval mode: The combination mode involves placing the images of multiple objects in sequence according to the spatial relationship triplets on a background canvas of a specified size (e.g., 1920×1080 pixels, RGB format) when there are multiple objects with spatial relationships.

[0078] The placement order is determined by spatial relationships. For example, larger objects are placed first as references, followed by smaller objects attached to them. During placement, the corresponding layout algorithm is called based on the relational terms in the triples (e.g., "above" means aligning the bottom of the second object with the top of the first object, and "beside" means horizontally adjacent and center-aligned, etc.).

[0079] The overall retrieval mode retrieves the corresponding complete background image directly from the library if the background environment encoding data contains only one overall scene description (such as a forest), skipping object-level combinations.

[0080] Regardless of the mode used, the bounding box coordinates (top left corner coordinates) of each object on the background canvas must be recorded. and the coordinates of the bottom right corner The background object position mapping table is formed as follows: ={object name: ( , ), ...}.

[0081] For the overall background image, the coordinates of the main objects within it need to be pre-labeled, and this labeling information is stored in the library as metadata.

[0082] It should be noted that the combination mode enables the fine-grained construction of complex scenes, ensuring that the spatial relationships between objects are consistent with the text description; while the overall retrieval mode simplifies the processing flow for simple scenes.

[0083] When combining objects, record the bounding box coordinates (i.e., position and size) of each object on the background canvas to form a background object position mapping table.

[0084] S3.3: Determine the initial reference point coordinates of the foreground subject based on the composition position.

[0085] In this embodiment of the invention, the specific operations include: Analyze the foreground-background spatial relationship data of the storyboard composition data, which contains one or more triples (foreground subject, relational word, background object). For each triple, perform the following operations: (1) For each spatial relationship triplet in the foreground-background spatial relationship data, obtain the bounding box coordinates of the corresponding background object from the background object position mapping table.

[0086] (2) Based on the preset relative position rules corresponding to the relation words and the initial reference point, calculate the coordinates of the candidate center point of the foreground subject that satisfies the spatial relationship: Assuming the size of the foreground subject is determined by the bounding box of the foreground image, but the foreground size is not considered when calculating the center point here. The coordinates of the top left corner need to be adjusted during actual placement, so the center point is calculated first: When the relation term is "above...", the vertical coordinate of the foreground center point is the upper boundary of the background object minus the foreground height / 2 (i.e., the bottom of the foreground touches the top of the background), and the horizontal center is aligned with the center of the background object; when the relation term is "below...", the vertical coordinate of the foreground center point is the lower boundary of the background object plus the foreground height / 2, and the horizontal center is aligned. When the relation term is "next to..." (default horizontally adjacent), the horizontal coordinate of the foreground center point is the right boundary of the background object plus the foreground width / 2, with vertical center alignment; or the left boundary minus the foreground width / 2, depending on the context. When the relation term is "inside...", the foreground center point is located inside the bounding box of the background object, adjusted according to the specific scene (e.g., when a person is in a house, the foreground needs to be completely within the background bounding box). When the relation term is "facing", the position is not changed, but it may affect the orientation selection of the foreground image (already processed during foreground retrieval), and the position calculation is ignored.

[0087] If the rule requires foreground size information, it can be obtained from the foreground image layer size. Since the final placement position has not yet been determined, the original width and height of the foreground image can be obtained first.

[0088] Furthermore, if multiple spatial relationship triples exist, a weighted average is calculated on the coordinates of all candidate center points (the preset weights can be set so that direct contact relationships have a higher weight than directional relationships; if the sum of the weights is 0, the arithmetic mean is taken), or one of them is selected according to a preset priority as the final foreground subject center point coordinates. The preset priority can be defined by relationship type (e.g., on... > next to... > near...), and the candidate point with the highest priority is taken as the final point; if the priorities are the same, the average is taken.

[0089] Using the initial reference point as a benchmark, select the candidate point that is closest to the initial reference point (with the smallest Euclidean distance) while satisfying all relational constraints (an exception will be prompted if there is a conflict).

[0090] Furthermore, if there is no foreground-background spatial relationship data, the initial reference point is used as the center point of the foreground subject, and the placement coordinates of the upper left corner of the foreground image on the background canvas are calculated.

[0091] The calculation process for the placement coordinates includes: Based on the finally determined foreground center point and the size (width) of the foreground image layer itself. ,high ), calculate the placement coordinates of the top-left corner of the foreground image on the background canvas. The calculation formula is: At the same time, it is necessary to check whether the placement coordinates exceed the boundaries of the background canvas.

[0092] If the boundary exceeds the limit, boundary constraint correction is performed. The boundary constraint correction process includes: shifting the foreground image inward to ensure that the complete image is within the canvas, and recording the correction offset.

[0093] If the shot is a close-up and the foreground subject is too large, the image size can be adjusted in advance during the retrieval stage (scaled according to the shot size), or the foreground image can be scaled before placement to ensure the subject occupies a suitable proportion. The scaling factor can be preset, for example, the subject height occupies 80% of the canvas height for a close-up and 30% for a medium shot.

[0094] As can be seen, this invention converts the center point coordinates into the upper left corner coordinates required for the actual placement of the image layer, which facilitates direct operation of the image synthesis library, and the boundary correction prevents partial image loss and ensures the integrity of the image.

[0095] Finally, the foreground image layer is placed according to the calculated coordinates. Overlay the image onto the background layer and use a foreground mask for pixel blending: replace background pixels with foreground pixels within the masked area, retain the background in the remaining areas, and output the blended storyboard. Example: Using the binary mask of the foreground during overlay Perform pixel-by-pixel blending: For each pixel of the background canvas : If the pixel is located within the coverage area of ​​the foreground image (i.e.) If M=255 (foreground region), the background pixels are replaced with foreground pixels; if M=0 (transparent region), the original background pixels are retained. If a pixel is not within the coverage area, the background is retained.

[0096] To improve the blending quality, anti-aliasing can be applied to the foreground edges, such as softening the mask, but this step is based on binary replacement to ensure determinism.

[0097] The final output is a complete blended storyboard in RGB format.

[0098] The merged storyboard images are associated with their corresponding shot parameters (duration, camera movement, etc.) to form a storyboard image data package with attributes. The process of processing all storyboard composition data within the same semantic narrative unit in S3 is repeated to obtain the storyboard image sequence for that unit.

[0099] As can be seen, the mask-guided pixel-level replacement of this invention ensures that the foreground subject accurately covers the background, avoiding background residue or color pollution that may be caused by alpha blending. The method is simple, efficient, and completely repeatable, making it suitable for batch processing. At the same time, it preserves unoccluded objects in the background, achieving correct occlusion relationships.

[0100] S4: Based on the duration and camera motion parameters of each scene, the sequence of scene images is synthesized into a video stream, and transition processing is performed to output the episode storyboard video file.

[0101] S4.1: Receives the fused frame of each shot output, along with the duration (in seconds) and shot motion parameters (values ​​include preset motion types such as stationary, zoom in, zoom out, panning, tracking, etc., or combinations thereof) from the corresponding shot parameters. Additionally, the shot parameters may contain motion-related information such as zoom level, translation direction, and target reference objects; this information can be extracted from the text parsing results or provided by preset default values.

[0102] S4.2: For each shot, generate a dynamic video clip based on the camera motion parameters. All dynamic clips are executed at a fixed frame rate. (For example, 24 frames per second or 30 frames per second) generated.

[0103] S4.2.1: If the camera movement is static, no transformation needs to be applied to the image. The generation method is: copy the storyboard frame. Next, get A sequence of images with identical frames, each frame lasting 1 / Seconds. This sequence is the static video clip of this storyboard.

[0104] S4.2.2: If the camera movement is a zoom in or zoom out, then continuous linear scaling of the image is required. First, determine the scaling factor. The scaling factor can be obtained in one of the following ways: Explicit multipliers extracted from text parsing (e.g., "double up") are stored in additional parameters; if no explicit description is provided, a default multiplier is used, such as 1.2x for zooming in (i.e., magnifying to 120%) and 0.8x for zooming out (i.e., shrinking to 80%).

[0105] For zooming in (magnification), the scaling factor changes linearly from 1.0 to... ( >1); For zooming in (reducing), the scaling factor changes linearly from 1.0 to (0< <1). Let the total number of frames be... Then the t-th frame (t from 0 to ...) Scaling factor of -1) (Push closer) or (When zoomed out) -1 is negative).

[0106] Applying scaling transformations to the original image may change the image size, requiring cropping or padding to maintain the same size as the original canvas. Typically, the scaling center is used as the scaling point. After scaling, the portion exceeding the canvas size is cropped, or the insufficient portion is filled with black (but usually, if the scaled size is larger than the canvas, cropping is done; if smaller, a black border is left. In practice, cropping is often used to simulate a camera zoom effect). Specifically: calculate the transformed image size, crop the area the size of the canvas using center alignment, or directly use image scaling library functions (such as OpenCV's `resize`) and set the cropping mode.

[0107] Each frame of the image is generated to form a scaled dynamic segment.

[0108] S4.2.3: If the camera movement type is panning (i.e., horizontal or vertical translation), then continuous linear translation of the image is required. First, determine the translation direction and distance. The translation direction can be extracted from the text description (e.g., panning left, panning up); if not specified, the default is horizontal. The translation distance can be determined in the following ways: The target reference position extracted from the text: For example, the tree swaying towards the door needs to be calculated according to the background object position mapping table recorded in S3 to calculate the expected position of the target reference in the picture (usually the target reference is centered), so as to obtain the translation vector; if there is no explicit reference, a preset translation distance is used, such as 20% of the width or 20% of the height of the picture.

[0109] Let the total number of frames be The starting frame is the original image (i.e., the current composition position), and the ending frame is the translated image. The starting coordinates of the translation are usually (0,0) (i.e., no image offset), and the ending offset is (dx, dy). Therefore, the offset of frame t is... .

[0110] The original image is translated and transformed. The part that exceeds the canvas is cropped, and the missing part is filled with black (or expanded according to the background content). However, panning usually means that the camera is rotating and the content of the image should be continuous. Therefore, the size of the original image remains unchanged during translation. This is achieved by moving the viewport: it is equivalent to moving on a larger virtual canvas. However, in actual implementation, the original image can be cropped and offset, that is, different areas of the original image are taken as the current frame each time, which is similar to the cropping effect of image translation.

[0111] The specific implementation process is as follows: the original image is regarded as a complete scene, and a sub-region of the same size as the canvas is cropped each time according to the offset. If the offset causes the boundary to exceed the limit, the excess part is filled with black.

[0112] Each frame of the image is generated to form a panning dynamic clip.

[0113] S4.2.3: If the camera movement type is other, such as tracking (following the subject's movement), vertical movement, or rotation, a corresponding affine transformation sequence must be generated based on preset transformation parameters or the motion path parsed from the text. Transformation parameters may include rotation angles, combined transformations, etc. The implementation is as follows: within the duration, predefined transformation matrices (such as a combination of rotation and translation matrices) are applied sequentially to the original image, with linear interpolation of the transformation parameters for each frame. For example, for a rotating shot, the rotation can be set from 0° to θ°. All transformations must ensure that the final output image size is consistent with the original canvas, and the boundaries are handled through cropping or filling.

[0114] If the motion description is complex and cannot be accurately parsed from the text, a simple default transformation (such as a small rotation) or ignoring the motion can be used. However, the system design should support extensions, allowing specific descriptions to be mapped to preset transformation sequences via a rule base.

[0115] S4.3: Gather all video clips (or static frames) generated from the storyboards within the same semantic narrative unit, arrange them according to the sentence order of the original storyboards, and form the video clip sequence of the semantic narrative unit. .

[0116] S4.4: Concatenate all the video segments after the transition processing in chronological order, encode them into a video stream, and output them as a storyboard video file for the TV series.

[0117] In practical operation, it is necessary to detect... Two adjacent segments ( and To ensure visual continuity at the transition points, a transitional frame is inserted based on the detection results. The specific method is as follows: (1) For The last frame (denoted as )and The first frame (denoted as Extract the binary mask of the foreground subject from the foreground mask data generated by S3 (or recalculate it). and The mask represents the pixel area where the foreground subject is located.

[0118] (2) Calculate the ratio of the intersection area (number of pixels) to the union area of ​​the two masks, i.e., the intersection-union ratio (IoU): If the denominator is 0 (i.e., there is no foreground in either frame), then IoU is considered to be 0.

[0119] (3) Set a preset overlap threshold (e.g., 0.3). If the IoU is less than the preset overlap threshold, it indicates that the position or content of the foreground subject in the preceding and following shots changes drastically, resulting in a strong visual jump. A transitional frame needs to be inserted to smooth the transition. The inserted transitional frame has a duration of [duration missing]. A black screen segment (e.g., 0.1 seconds), which is an image sequence in which all pixels have a value of 0, with a frame count of... Insert the black screen segment into and between.

[0120] If the IoU is greater than or equal to the preset overlap threshold, no insertion is needed, and the connection can be made directly.

[0121] Finally, the above detection and insertion are performed on all adjacent pairs in the sequence to obtain a new sequence of transitioned video segments.

[0122] All segments in the transitioned video sequence are sequentially concatenated to form a complete frame sequence. This sequence includes frames from all scenes and inserted transition frames, with the total number of frames equal to the sum of the frames of each segment. This frame sequence is then encoded into a video file. Encoding parameters can be preset, such as: video encoding format H.264 or H.265; frame rate. (Same as when it was generated); the resolution is the same as the screen size of the storyboard (e.g., 1920×1080); the bitrate is set according to the image quality requirements (e.g., 10Mbps).

[0123] The frame sequence is written to a file using a standard video encoding library (such as FFmpeg), and the output is a storyboard video file for the series. The file name can be automatically generated based on the script name or unit number.

[0124] Furthermore, such as Figure 2 As shown, the present invention also provides an automatic generation system for TV series storyboard based on shot semantic decomposition, including: a script unit division module, which receives the script and divides the script into several continuous semantic narrative units based on scene switching markers and plot emotion curves in the script. The storyboard generation module parses the text description contained in each semantic narrative unit and generates a corresponding storyboard sequence, which includes shot parameters and a description of the scene composition. The image retrieval and fusion module retrieves foreground and background images from a preset image library based on the storyboard sequence and fuses them into a storyboard frame. The video compositing output module combines the sequence of scene shots into a video stream based on the duration and camera motion parameters of each scene, performs transition processing, and outputs the episode's scene video file.

[0125] The system also includes one or more processors and memory.

[0126] The memory is used to store operable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, including the flow of an automatic storyboard generation method for TV series based on shot semantic decomposition as described in the foregoing embodiments, particularly... Figure 1 The flowchart of the method is shown.

[0127] Other aspects disclosed in the embodiments of the present invention also propose a computer-readable medium for storing software including instructions executable by one or more computers, which, upon execution, cause the one or more computers to perform operations including the flow of an automatic storyboard generation method for TV series based on shot semantic decomposition as described in the foregoing embodiments, particularly... Figure 1 The flowchart of the method is shown.

[0128] It should be recognized that embodiments of the present invention may be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium.

[0129] The method can be implemented using standard programming techniques, including a non-transitory computer-readable storage medium configured with a computer program in the computer program, wherein the storage medium is configured such that the computer operates in a specific and predefined manner.

[0130] Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system; however, if required, the program can be implemented in assembly or machine language.

[0131] In any case, the language can be either compiled or interpreted.

[0132] Furthermore, for this purpose, the program can run on programmed application-specific integrated circuits.

[0133] The processes described herein (or variations and / or combinations thereof) can be executed under the control of one or more computer systems configured with executable instructions, and can be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. The computer program includes a plurality of instructions executable by one or more processors.

[0134] Furthermore, the method can be implemented in any suitable computing platform, including but not limited to personal computers, minicomputers, mainframes, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or in communication with charged particle tools or other imaging devices.

[0135] Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether portable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein.

[0136] Furthermore, machine-readable code, or parts thereof, can be transmitted via wired or wireless networks.

[0137] When such media includes instructions or programs that combine with a microprocessor or other data processor to implement the steps described above, the invention described herein includes these and other different types of non-transitory computer-readable storage media.

[0138] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for automatically generating storyboards for TV series based on shot semantic decomposition, characterized in that: include: Upon receiving the script, based on scene transition markers and emotional curves in the script, the script is divided into several continuous semantic narrative units. For each semantic narrative unit, the contained text description is parsed to generate a corresponding storyboard sequence, which includes shot parameters and a description of the scene composition; Based on the storyboard sequence, foreground and background images are retrieved from a preset image library and merged into a storyboard frame; Based on the duration and camera motion parameters of each scene, the sequence of scene images is synthesized into a video stream, and transition processing is performed to output the episode's scene video file.

2. The automatic generation method for TV series storyboards based on shot semantic decomposition as described in claim 1, characterized in that: The division of the semantic narrative unit includes: The script is analyzed, scene transition markers are extracted, and the script is divided into multiple scene units based on the scene transition markers. Each scene unit corresponds to a continuous plot. For each scene unit, extract the text content and count the number of positive and negative sentiment words in the preset sentiment word library for each sentence. The difference between the number of positive sentiment words and the number of negative sentiment words is used as the sentiment intensity value of the corresponding text content to generate the sentiment intensity sequence of the scene unit; If there are no emotion words, the emotion intensity value is set to 0.

3. The automatic generation method for TV series storyboards based on shot semantic decomposition as described in claim 2, characterized in that: The division of semantic narrative units also includes: Identify local minima in the emotional intensity sequence; Using the local minimum point as the segmentation boundary, each scene unit is further divided into at least one semantic narrative unit, and each semantic narrative unit contains a complete emotional change range from the peak value to the trough value of emotional intensity. If the emotional intensity sequence has no local minimum, then the scene unit as a whole is treated as a semantic narrative unit.

4. The automatic generation method for TV series storyboards based on shot semantic decomposition as described in claim 3, characterized in that: The parsing includes text descriptions, generating corresponding storyboard sequences including: The semantic narrative unit is received, and the text is split into several clauses according to sentences. Each clause corresponds to a potential scene. For each clause, the noun as the subject is extracted as the core foreground candidate, the verb phrase as the predicate or accompanying adverbial is extracted as the action description, and the noun as the object or prepositional object is extracted as the background object candidate. For each clause, the core foreground candidate is determined as the core foreground subject, and it is checked whether the core foreground candidate is modified by a preset emphasis modifier. If it is modified, the shot type in the shot parameters of the scene is marked as a close-up; otherwise, the shot type is defaulted to medium shot. Other nouns not selected as core foreground elements are categorized under background context information. For each clause's core foreground subject, the verb corresponding to the core foreground subject is extracted from the action description as the action type, the adverb modifying the verb is extracted as the action modifier, and the modifier describing the subject's posture is also extracted. The above information is combined to form foreground subject encoding data. For each object noun in the background environment information of each clause, prepositional phrases describing the spatial relationship between objects are extracted from the text. Based on the prepositional phrases, object spatial relationship triples are generated. The triples include the first object, the spatial relationship word, and the second object. The object nouns and the object spatial relationship triples are summarized into background environment encoding data.

5. The automatic generation method for TV series storyboards based on shot semantic decomposition as described in claim 4, characterized in that: The parsing includes text descriptions, and generating corresponding storyboard sequence packages also includes: Extract the prepositional phrases describing the spatial relationship between the core foreground subject and the background environment from each clause to generate foreground-background spatial relationship data, which contains one or more spatial relationship triples. If there is no explicit spatial relationship description in the text, it is assumed that the foreground subject is located in the center of the image and has no specific background association. Extract shot motion descriptions from each clause as shot motion parameters; if none are found, default to stillness. At the same time, determine composition position parameters based on composition descriptions appearing in the text; if none are found, set a default composition based on the shot size. The foreground subject encoding data, background environment encoding data, foreground-background spatial relationship data, and determined shot parameters corresponding to each clause are combined to form the shot composition data corresponding to the current clause; Arrange the storyboard data corresponding to all clauses within the same semantic narrative unit in clause order, and output the storyboard sequence of the semantic narrative unit.

6. The automatic generation method for TV series storyboards based on shot semantic decomposition as described in claim 5, characterized in that: The fusion into a storyboard includes: Based on the foreground subject encoding data, a matching image is retrieved from the preset character image library. If multiple angles or pose variations exist, the image that best matches the description is selected. If no perfect match is found, the default pose image is selected and labeled with a mask. The foreground image and its corresponding binary mask are then output. Based on the object names and spatial relationship triples in the background environment coding data, the image of each object is retrieved from the preset scene object library and combined on the blank canvas according to the spatial relationship triples to generate the background image layer. During the combination process, the bounding box coordinates of each object on the background canvas are recorded to form a background object position mapping table.

7. The automatic generation method for TV series storyboards based on shot semantic decomposition as described in claim 6, characterized in that: The fusion into a storyboard also includes: Determine the initial reference point coordinates of the foreground subject based on its position in the composition; For each spatial relationship triple in the foreground-background spatial relationship data, obtain the bounding box coordinates of the corresponding background object from the background object position mapping table; Based on the preset relative position rules corresponding to the relation words, calculate the coordinates of the candidate center point of the foreground subject that satisfies the spatial relationship; If multiple spatial relationship triples exist, the coordinates of all candidate center points are weighted and averaged, or one of them is selected according to a preset priority as the final coordinates of the foreground subject center point. If there is no foreground-background spatial relationship data, then the initial reference point is used as the center point of the foreground subject, and the placement coordinates of the upper left corner of the foreground image on the background canvas are calculated. The foreground image layer is overlaid on the background image layer according to the calculated placement coordinates. Pixel blending is performed using a foreground mask: foreground pixels are used to replace background pixels within the mask area, while the background is retained in the remaining areas. The blended storyboard is then output.

8. The automatic generation method for TV series storyboards based on shot semantic decomposition as described in claim 7, characterized in that: The step of synthesizing the sequence of storyboard frames into a video stream includes: Receive the fused image of each shot output, as well as the duration and camera motion parameters in the corresponding shot parameters; set a basic duration for each shot, but if the camera motion parameters are not static, a dynamic image needs to be generated within the basic duration. For each shot, a dynamic video clip is generated based on the camera motion parameters; All video clips generated from storyboards within the same semantic narrative unit are collected and arranged in sentence order to form a video clip sequence for the semantic narrative unit. All the processed video clips are spliced ​​together in chronological order, encoded into a video stream, and output as a storyboard video file for the TV series.

9. The automatic generation method for TV series storyboards based on shot semantic decomposition as described in claim 8, characterized in that: The process of generating dynamic video clips based on camera motion parameters includes: If the camera movement is stationary, the merged image is directly used as the static frame of the segment duration; If the camera movement is a zoom in or zoom out, the merged image is continuously and linearly scaled within the duration of the scene according to a preset scaling factor or a scaling factor extracted from the text, generating a scaled dynamic segment. If the camera movement is a panning shot, the merged image is continuously and linearly panned within the duration of the segment based on a preset translation distance or the position of the target reference extracted from the text, generating a panning dynamic segment; the starting point of the translation is the current composition position, and the ending point is the corresponding position determined according to the preset translation distance, or the expected position of the target reference in the image; If the camera movement is of another type, then the corresponding affine transformation dynamic segment is generated according to the transformation path determined by the preset transformation parameters or text semantics. All dynamic clips are generated at a fixed frame rate.

10. A system for automatically generating storyboards for TV series based on shot semantic decomposition, based on the method for automatically generating storyboards for TV series based on shot semantic decomposition as described in any one of claims 1 to 9, characterized in that: Also includes: The script unit division module receives the script and, based on scene transition markers and plot emotion curves in the script, divides the script into several continuous semantic narrative units. The storyboard generation module parses the text description contained in each semantic narrative unit and generates a corresponding storyboard sequence, which includes shot parameters and a description of the scene composition. The image retrieval and fusion module retrieves foreground and background images from a preset image library based on the storyboard sequence and fuses them into a storyboard frame. The video compositing output module combines the sequence of scene shots into a video stream based on the duration and camera motion parameters of each scene, performs transition processing, and outputs the episode's scene video file.

Images