A method for generating and managing movie shots based on generative artificial intelligence

Abstract

The application discloses a kind of based on generative artificial intelligence's film and television shot generation and management method, including the following steps: S1, collect film and television creation input information, construct shot generation demand description set;S2, execute semantic analysis processing, generate shot generation intent structure;S3, construct shot generation condition set;S4, input to improved Sora model, construct shot state anchor point and introduce state conservation check mechanism, generate candidate shot set;S5, execute consistency evaluation processing, generate shot state description;S6, construct shot management unit, generate shot evolution record;S7, output film and television shot result, realize the generation and management of film and television shot.The application improves the overall quality of shot generation result in narrative consistency, emotional coherence and visual attention stability, and enhances the traceability and management efficiency of film and television shot result.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and film and television content generation technology, and in particular to a method for generating and managing film and television shots based on generative artificial intelligence. Background Technology

[0002] With the continuous improvement of digitalization and intelligence in the film and television industry, AI-based film and television content generation technologies are gradually being applied to script visualization, storyboard design, and shot previewing. Currently, the generation of film and television shots typically relies on manual storyboard experience or text-to-video generation models, directly outputting continuous images based on the script description to assist directors or production teams in making creative decisions. While these technologies have reduced pre-production costs and improved creative efficiency to some extent, they still reveal significant shortcomings in complex narrative structures and scenarios requiring meticulous shot management.

[0003] On the one hand, existing methods for generating film and television shots often focus on the generated content, lacking structured modeling of the shot's inherent state, such as narrative progression, emotional changes, and visual attention. The generated results rely on implicit model learning for temporal continuity and narrative consistency, making them prone to issues like semantic jumps, emotional breaks, or subject drift, failing to meet the requirements of professional film and television production for shot stability and expressive accuracy. On the other hand, existing technologies often output shots directly as results, lacking state descriptions and evolutionary records of the shot generation process. This makes it difficult to trace the basis for generation and state changes during subsequent editing, reuse, and management, limiting the asset management and systematic iterative application of shots.

[0004] Furthermore, regarding the quality control of the generated results, most existing solutions rely solely on simple manual screening or overall scoring methods for evaluation. They lack fine-grained consistency assessment methods based on the intention of shot generation, and are unable to effectively constrain and verify the narrative, emotional, and visual attention states during the generation process. This affects the controllability and reliability of the generated shots in the actual film and television production process.

[0005] Therefore, how to provide a method for generating and managing film and television shots based on generative artificial intelligence is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a method for generating and managing film and television shots based on generative artificial intelligence. This invention addresses the problems of inconsistent expression of shot generation requirements, insufficient consistency in generation results, and lack of manageable evolutionary records for shot results during film and television creation. It constructs a data processing flow centered on a set of shot generation requirement descriptions, a shot generation intention structure, and a set of shot generation conditions. An improved Sora model is introduced to construct shot state anchor points and a state conservation verification mechanism. This model completes the generation of candidate shot sets, consistency evaluation, and construction of shot state descriptions. Simultaneously, a shot management unit uniformly registers, generates, and associates evolutionary records of candidate shots and their corresponding shot state descriptions, achieving orderly output and management of film and television shot results. This invention achieves controllable shot generation processes, traceable shot states, and manageable shot results while ensuring consistency between the shot generation results and the shot generation intention structure. It possesses the advantages of high generation consistency, clear evolutionary processes, and strong management capabilities.

[0007] A method for generating and managing film and television shots based on generative artificial intelligence according to an embodiment of the present invention includes the following steps:

[0008] S1. Collect input information for film and television creation and construct a set of shot generation requirements descriptions;

[0009] S2. Perform semantic parsing processing on the shot generation requirement description set to generate a shot generation intent structure;

[0010] S3. Construct a set of shot generation conditions based on the shot generation intention structure;

[0011] S4. Input the shot generation intention structure and the shot generation condition set into the improved Sora model. The improved Sora model constructs shot state anchor points and introduces a state conservation verification mechanism to generate a candidate shot set.

[0012] S5. Perform a consistency evaluation process on the candidate shot set based on the shot generation intention structure to generate a shot state description;

[0013] S6. Construct a lens management unit based on the lens state description, input the candidate lenses in the candidate lens set and their corresponding lens state descriptions into the lens management unit, and generate a lens evolution record.

[0014] S7. Based on the shot evolution record, output the film and television shot results from the shot management unit, and associate and store the film and television shot results with the corresponding shot evolution record to realize the generation and management of film and television shots.

[0015] Optionally, S1 specifically includes:

[0016] Collect script excerpts, character settings, scene settings, emotional tone, and visual style information;

[0017] The script fragment information is processed by sentence segmentation to obtain a set of narrative fragments; the character setting information is processed by entity normalization to obtain a set of character entities; the scene setting information is processed by scene tagging to obtain a set of scene tags; the emotional tone information is processed by emotional tagging to obtain a set of emotional tags; and the visual style information is processed by style tagging to obtain a set of style tags.

[0018] Based on the set of narrative fragments, character entities, scene tags, emotion tags, and style tags, a structured splicing process is performed to construct a set of shot generation requirements descriptions.

[0019] Optionally, S2 specifically includes:

[0020] The shot generation requirement description set is processed by word segmentation and syntactic analysis to obtain a sequence of semantic units and a sequence of dependency relations.

[0021] Based on the semantic unit sequence, event element extraction processing is performed to obtain the action word set, subject word set, object word set and modifier word set;

[0022] Based on the dependency relation sequence, the action word set, subject word set, object word set, and modifier set are combined to obtain the narrative event set;

[0023] Perform shot intent tag mapping on the narrative event set to generate a shot function tag set, an emotion tag set, a narrative progression tag set, and a visual attention tag set;

[0024] Based on the set of shot function labels, set of emotion labels, set of narrative progression labels, and set of visual attention labels, vectorization processing is performed and the data is spliced ​​to generate a shot generation intention structure.

[0025] Optionally, S3 specifically includes:

[0026] Read the set of shot function tags, set of emotion tags, set of narrative progression tags, and set of visual attention tags in the shot generation intention structure;

[0027] Perform shot type mapping processing on the shot function tag set to generate shot type conditions; perform emotion intensity calibration processing on the emotion tag set to generate emotion conditions; perform event sequence calibration processing on the narrative progression tag set to generate narrative conditions; perform subject focus calibration processing on the visual attention tag set to generate attention conditions.

[0028] The conditions for shot generation are spliced ​​together by performing condition splicing processing on shot type conditions, emotional conditions, narrative conditions and attention conditions to construct a set of shot generation conditions.

[0029] Optionally, S4 specifically includes:

[0030] Read the shot generation intention structure and shot generation condition set;

[0031] Based on the shot generation intention structure, extract the narrative progression tag set, emotion tag set, and visual attention tag set, and construct the shot state anchor point set;

[0032] The set of shot state anchor points and the set of shot generation conditions are combined to generate an input sequence, and the input sequence is fed into the improved Sora model.

[0033] Candidate shots are generated based on the improved Sora model, and state extraction processing is performed on the candidate shots to obtain narrative state sequence, emotional state sequence and visual subject state sequence.

[0034] State conservation verification is performed on the narrative state sequence, emotional state sequence and visual subject state sequence. If the offset of any state sequence is greater than the preset range, regeneration is performed based on the input sequence.

[0035] The output of the state conservation verification process is used to collect the generated candidate shots and form a candidate shot set.

[0036] Optionally, the improved Sora model is specifically as follows:

[0037] Receive an input sequence consisting of a shot generation intention structure and a shot generation condition set, wherein the input sequence is arranged in chronological order;

[0038] The condition set of shots in the input sequence is generated by performing vector mapping processing to obtain a condition vector sequence;

[0039] Perform state encoding processing on the set of shot state anchor points in the input sequence to obtain an anchor point vector sequence;

[0040] The condition vector sequence and the anchor vector sequence are aligned, and the aligned vectors are concatenated to form a joint control vector sequence.

[0041] Temporal reasoning is performed based on the joint control vector sequence to generate a continuous shot frame sequence, which constitutes a candidate shot.

[0042] Feature extraction processing is performed on a continuous sequence of shot frames to obtain a narrative state sequence, an emotional state sequence, and a visual subject state sequence.

[0043] The state sequence offset is obtained by calculating the state difference between adjacent frames for the narrative state sequence, the emotional state sequence, and the visual subject state sequence.

[0044] The state sequence offset is compared with the preset range to obtain the state sequence offset result;

[0045] If the state sequence offset result exceeds the preset range, the joint control vector sequence is adjusted based on the input sequence and the temporal inference generation is completed again;

[0046] If the state sequence offset result is within a preset range, then a set of candidate shots will be output.

[0047] Optionally, the step of extracting a set of narrative progression tags, an emotion tag set, and a visual attention tag set based on the shot generation intention structure to construct a set of shot state anchor points specifically involves:

[0048] Extract the corresponding event sequence identifier for each narrative advancement tag in the narrative advancement tag set, and sort the narrative advancement tag set according to the event sequence identifier to generate a narrative advancement tag sequence;

[0049] Extract the corresponding emotion change sequence identifier for each emotion tag in the emotion tag set, and sort the emotion tag set according to the emotion change sequence identifier to generate an emotion tag sequence.

[0050] For each visual attention tag in the visual attention tag set, extract the corresponding subject identifier and spatial attention order identifier, and sort the visual attention tag set according to the spatial attention order identifier to generate a visual attention tag sequence.

[0051] Based on the sequence length of the narrative progression tag sequence, the emotion tag sequence and the visual attention tag sequence are index aligned according to their index positions to generate multiple aligned tag groups;

[0052] Perform a combination process on each aligned tag group to form a combined tag unit that includes narrative progression tags, emotion tags, and visual attention tags;

[0053] Arrange multiple combined label units in sequence to generate a set of lens state anchor points.

[0054] Optionally, S5 specifically includes:

[0055] For each candidate shot in the candidate shot set, extract the corresponding narrative state sequence, emotional state sequence, and visual subject state sequence;

[0056] Extract the set of narrative progression tags, the set of emotion tags, and the set of visual attention tags from the shot generation intention structure;

[0057] Based on the narrative advancement tag set, state matching processing is performed on the narrative state sequence to obtain narrative consistency results;

[0058] Based on the set of emotion labels, state matching processing is performed on the emotion state sequence to obtain emotion consistency results;

[0059] State matching processing is performed on the visual subject state sequence based on the visual attention tag set to obtain attention consistency results;

[0060] The narrative consistency result, emotional consistency result, and attention consistency result are fused to generate the shot consistency result; a shot state description is generated based on the shot consistency result.

[0061] Optionally, S6 specifically includes:

[0062] Assign a lens identifier to each candidate lens in the candidate lens set, and build a lens index table based on the lens identifier;

[0063] Write the lens identifier and corresponding lens status description into the lens index table to generate a lens registration record; write a time identifier into the lens registration record and establish an evolution sequence table based on the time identifier;

[0064] The candidate shots are associated with the shot registration records and stored to generate shot storage records;

[0065] A lens evolution record is generated based on lens identifiers, time identifiers, lens storage records, and lens state descriptions.

[0066] Optionally, S7 specifically includes:

[0067] Lens identifiers, time identifiers, and lens storage records are extracted from the lens evolution record; the lens storage records are sorted according to the time identifiers to generate an output sequence;

[0068] Candidate shots are extracted from the shot storage record based on the output sequence to generate film and television shot results;

[0069] Write the video shot results into the output record and associate the output record with the shot identifier; perform associated storage processing on the video shot results and shot evolution record to generate management storage record.

[0070] The beneficial effects of this invention are:

[0071] This invention constructs a set of shot generation requirements, a shot generation intention structure, and a set of shot generation conditions to structure the input information for film and television creation. It transforms key information such as narrative progression, emotional expression, and visual attention into a computable, unified expression. Furthermore, it introduces an improved Sora model during the shot generation stage, which constructs shot state anchors and a state conservation verification mechanism. This model continuously constrains and verifies the narrative state sequence, emotional state sequence, and visual subject state sequence during candidate shot generation, effectively suppressing narrative deviation, emotional discontinuity, and visual attention drift during shot generation. Simultaneously, this invention uses the shot generation intention structure to perform structural analysis on candidate shots... The head set performs consistency evaluation processing to form a shot state description, enabling the shot generation results to have quantifiable and comparable state representations in terms of narrative consistency, emotional consistency, and attention consistency. In the shot management stage, by constructing the association between shot identifiers, time identifiers, and shot evolution records, the unified registration, sequential evolution, and associated storage of candidate shots and shot state descriptions are realized, ensuring the traceability and manageability of the film and television shot output process. This invention achieves the technical effects of controllable generation process, high consistency of generation results, and complete and clear evolution records in the entire process of film and television shot generation and management, significantly improving the quality of film and television shot generation and subsequent management efficiency. Attached Figure Description

[0072] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0073] Figure 1 This is a flowchart of a film and television shot generation and management method based on generative artificial intelligence proposed in this invention;

[0074] Figure 2 This is a schematic diagram of the structure of the improved Sora model proposed in this invention;

[0075] Figure 3 This is a data flow diagram of a film and television shot generation and management method based on generative artificial intelligence proposed in this invention. Detailed Implementation

[0076] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0077] refer to Figure 1-3 A method for generating and managing film and television shots based on generative artificial intelligence includes the following steps:

[0078] S1. Collect input information for film and television creation and construct a set of shot generation requirements descriptions;

[0079] S2. Perform semantic parsing processing on the shot generation requirement description set to generate a shot generation intent structure;

[0080] S3. Construct a set of shot generation conditions based on the shot generation intention structure;

[0081] S4. Input the shot generation intention structure and the shot generation condition set into the improved Sora model. The improved Sora model constructs shot state anchor points and introduces a state conservation verification mechanism to generate a candidate shot set.

[0082] S5. Perform a consistency evaluation process on the candidate shot set based on the shot generation intention structure to generate a shot state description;

[0083] S6. Construct a lens management unit based on the lens state description, input the candidate lenses in the candidate lens set and their corresponding lens state descriptions into the lens management unit, and generate a lens evolution record.

[0084] S7. Based on the shot evolution record, output the film and television shot results from the shot management unit, and associate and store the film and television shot results with the corresponding shot evolution record to realize the generation and management of film and television shots.

[0085] This implementation method collects film and television creation input information and constructs a shot generation requirement description set, enabling the film and television creation intent to be expressed in a structured and unified form, reducing the ambiguity of input information and improving the consistency of subsequent processing. Furthermore, by performing semantic parsing processing on the shot generation requirement description set and generating a shot generation intent structure, it achieves a centralized representation of narrative goals, emotional expression, and visual attention elements, enhancing the shot generation process's understanding of creative intent. Further still, based on the shot generation intent structure, a shot generation condition set is constructed, clearly constraining shot type, emotional intensity, narrative order, and subject attention, thereby improving the controllability of generated shots in terms of content and form. Simultaneously, the shot generation intent structure and shot generation condition set are input into an improved Sora model, and shot state anchor points are constructed and state conservation checks are introduced. The mechanism ensures the continuity and stability of the narrative, emotional, and visual subject states throughout the generation process, reducing shot jumps and semantic drift, and obtaining a consistent set of candidate shots. Furthermore, based on the shot generation intention structure, a consistency evaluation process is performed on the candidate shot set, generating shot state descriptions. This establishes a quantitative correspondence between the generated results and the original creative intentions, enhancing the reliability of shot selection and management. Simultaneously, shot management units are constructed based on the shot state descriptions, generating shot evolution records. This ensures that the generated shots have traceable evolutionary relationships and chronological order, improving shot organization and version management capabilities. Finally, based on the shot evolution records, the film and television shot results are output and stored in association, creating a closed loop between the film and television shot generation and management processes. This comprehensively improves the stability, controllability, and management efficiency of film and television shot generation.

[0086] In this embodiment, S1 specifically refers to:

[0087] Collect script excerpt information, character setting information, scene setting information, emotional tone information, and visual style information. Script excerpt information is entered in the form of text paragraphs, character setting information is entered in the form of character name, identity attributes, and relationship description, scene setting information is entered in the form of spatial location, environmental elements, and time attributes, emotional tone information is entered in the form of emotion category and intensity description, and visual style information is entered in the form of color tendency, composition preference, and visual style description.

[0088] The script excerpt information is processed into sentences according to punctuation and semantic pause rules. Sentence segmentation is performed by detecting periods, question marks, exclamation marks and semicolons.

[0089] The character setting information is subjected to entity normalization processing. Entity normalization processing is completed by mapping the character names with different referential forms to a unified character identifier. The character identifier consists of a character number and a character name. The character identifier number adopts a continuous positive integer encoding rule to obtain a set of character entities.

[0090] Scene tagging is performed on the scene setting information. Scene tagging is done by splitting the scene setting information into location tags, time tags, and environment tags. Location tags include categories such as indoor, outdoor, and specific building names. Time tags include categories such as daytime, nighttime, and early morning. Environment tags include weather conditions and spatial characteristics, resulting in a set of scene tags.

[0091] Emotional tone information is processed by emotion labeling. Emotional labeling is completed by mapping emotional tone information to emotion category labels and emotion intensity labels. Emotional category labels include basic emotion types such as calm, tension, joy, and sadness. Emotional intensity labels are represented by integer levels from one to five, with higher values ​​indicating higher emotion intensity, resulting in a set of emotion labels.

[0092] The visual style information is processed by style tagging. Style tagging is completed by mapping the visual style information into color style tags, shot composition tags and picture rhythm tags. Color style tags include cool, warm and neutral color categories, shot composition tags include close-up, medium and long shot categories, and picture rhythm tags include slow, medium and fast categories, resulting in a set of style tags.

[0093] Based on the set of narrative fragments, character entities, scene tags, emotion tags, and style tags, a structured splicing process is performed in a fixed field order. The structured splicing process is completed by associating each narrative fragment with corresponding character entities, scene tags, emotion tags, and style tags. The splicing result is stored in the form of a multi-field structured record, generating a shot generation requirement description set.

[0094] In this embodiment, S2 specifically refers to:

[0095] The text fields in each structured record of the shot generation requirement description set are scanned character by character and word boundaries are determined. Word segmentation is completed by matching a pre-set vocabulary and part-of-speech rules. The word segmentation results are arranged in the original text order to form a semantic unit sequence. Each semantic unit in the semantic unit sequence contains at least word text content and part-of-speech tag. At the same time, based on the part-of-speech tag and word order relationship in the semantic unit sequence, the dependency relationship between words is established. The dependency relationship is formed by recording the position index of the dominant word and the position index of the subordinate word, resulting in a dependency relationship sequence.

[0096] Based on the semantic unit sequence, each semantic unit is classified and filtered according to its part-of-speech tag. Semantic units with verb parts of speech are assigned to the action word set, semantic units with noun or pronoun parts of speech that meet the conditions of referring to a person or object are assigned to the subject word set or the object word set, and semantic units with adjective, adverb or preposition parts of speech that describe the manner, state or environmental characteristics of an action are assigned to the modifier word set. Preferably, the number of action words in a single narrative segment is set to one to five, and the number of subject words is set to one to three.

[0097] Based on the dependency sequence, semantic units in the action word set, subject word set, object word set and modifier set are associated and combined according to the dependency pointing relationship. The association and combination process is completed by combining the subject word, object word and modifier associated with the same action word index into an event unit. Each event unit contains action elements, execution subject, target and modification information. Multiple event units are arranged in the order of text appearance to obtain a narrative event set.

[0098] For each event unit in the narrative event set, a preset tag mapping rule is matched based on the action elements and modification information in the event unit to complete the shot intention tag mapping process. The tag mapping rule is determined by corresponding the event functional attributes, emotional expression attributes, narrative position attributes and visual attention attributes. Among them, the event functional attributes are mapped to shot function tags, the emotional expression attributes are mapped to emotional tags, the narrative position attributes are mapped to narrative progression tags, and the visual attention attributes are mapped to visual attention tags. Preferably, the narrative progression tags use consecutive integers to represent the sequential position of the event in the narrative sequence.

[0099] Based on the set of camera function labels, the set of emotion labels, the set of narrative progression labels, and the set of visual attention labels, numerical encoding processing is performed on each type of label. The numerical encoding processing is completed by assigning a fixed-length numerical vector to different label categories. The emotion label vector contains emotion category code and emotion intensity code. The emotion intensity code is represented by integers from one to five.

[0100] The shot function label vector, emotion label vector, narrative progression label vector, and visual attention label vector are spliced ​​together in a predetermined order, and the splicing result forms a vector representation of uniform length, which serves as the shot generation intention structure.

[0101] In this embodiment, S3 specifically refers to:

[0102] The shot generation intention structure is read item by item from the shot function tag set, emotion tag set, narrative progression tag set and visual attention tag set. The shot function tag set is used to characterize the functional attributes of the shot in the narrative, the emotion tag set is used to characterize the emotion category and intensity that the shot needs to present, the narrative progression tag set is used to characterize the position and order of the shot in the overall narrative sequence, and the visual attention tag set is used to characterize the main object that needs to be highlighted in the picture.

[0103] For the lens function label set, the function category is mapped to the corresponding lens type condition according to the function category identifier recorded in the lens function label. The lens type condition is represented in the form of discrete categories, including close-up, medium shot, long shot, moving shot and static shot, etc. Preferably, each lens type in the lens type condition corresponds to a fixed number, which is used for subsequent condition combination and generation control.

[0104] For the set of emotion tags, the emotion intensity is calibrated based on the emotion category code and emotion intensity code recorded in the emotion tags. The emotion intensity calibration determines the emotional condition of the shot by reading the value of the emotion intensity code. The emotion intensity code is represented by an integer from one to five, where one represents a low intensity emotion and five represents a high intensity emotion. Preferably, when the emotion intensity value is greater than or equal to three, it is determined to be a strong emotion condition, and when it is less than three, it is determined to be a weak emotion condition, and the emotion condition is generated.

[0105] For the narrative progression tag set, the event sequence is determined based on the sequence index value recorded in the narrative progression tag. The event sequence determination is used to limit the sequential position of the shots in the generated sequence by directly using the sequence index value as part of the narrative conditions. Preferably, the sequence index value is represented by consecutive positive integers, and the smaller the value, the earlier the narrative position is, thus generating the narrative conditions.

[0106] For the set of visual attention tags, subject focus calibration is completed based on the subject identifier and spatial attention identifier recorded in the visual attention tags. Subject focus calibration generates attention conditions by determining the subject objects that need to be clearly presented in the picture and the spatial area where the subjects are located. Preferably, the number of subjects in subject focus calibration is set to one or two, and the spatial attention range is represented by the central area of ​​the picture or one-third of the composition area of ​​the picture.

[0107] Based on shot type conditions, emotional conditions, narrative conditions, and attention conditions, condition splicing processing is performed according to the preset field order. Condition splicing processing is completed by combining each condition in a fixed order into a multi-field condition record. Multiple condition records form a shot generation condition set, which is used to constrain the shot form, emotional presentation, narrative position, and visual subject expression in the subsequent shot generation process.

[0108] In this embodiment, S4 specifically refers to:

[0109] The corresponding vectorized representation is read from the stored shot generation intention structure, and the multi-field condition record consisting of shot type condition, emotion condition, narrative condition and attention condition is read from the shot generation condition set. The shot generation intention structure and the shot generation condition set maintain a one-to-one correspondence in the time index dimension.

[0110] Based on the narrative progression tag set, emotion tag set, and visual attention tag set in the shot generation intention structure, the tags are sorted according to the sequential index value recorded in the narrative progression tag, and the corresponding emotion tag and visual attention tag are combined at each sequential index position to form a shot state anchor point. The shot state anchor point is used to describe the narrative state, emotional state, and visual subject state at the same narrative position. Multiple shot state anchor points are arranged in narrative order to form a shot state anchor point set. Preferably, a single shot state anchor point contains one narrative progression tag, one emotion tag, and one visual attention tag.

[0111] Based on the set of shot state anchor points and the set of shot generation conditions, the shot state anchor points and the corresponding shot generation conditions are combined at the field level under the same narrative order index. The field-level combination is formed by concatenating the state anchor point vector and the condition vector in a fixed order. Multiple concatenation results are arranged in chronological order to generate an input sequence, which serves as the input data for the improved Sora model.

[0112] The input sequence is fed into the improved Sora model. During the generation process, the improved Sora model controls the temporal consistency of the video frame generation process based on the narrative position constraints, emotional intensity constraints and visual subject constraints contained in the input sequence, and outputs a continuous video frame sequence. The continuous video frame sequence is divided into multiple candidate shots according to a preset frame window.

[0113] For each candidate shot, video frame sequence is used to extract narrative-related features, emotion-related features, and visual subject-related features. Narrative-related features are obtained by analyzing changes in action and continuity of events in the scene. Emotion-related features are obtained by analyzing color distribution, brightness changes, and facial expression features in the scene. Visual subject-related features are obtained by analyzing the position and clarity of the main object in the scene. The features are arranged in chronological order to form narrative state sequence, emotion state sequence, and visual subject state sequence, respectively.

[0114] Based on the narrative state sequence, emotional state sequence, and visual subject state sequence, the magnitude of state change at adjacent time positions is calculated. The magnitude of state change is represented by the difference between the state feature value of the previous time position and the state feature value of the next time position, and the difference is represented by a normalized numerical value.

[0115] The magnitude of state change is compared with a preset range. The preset range is used to limit the allowable degree of change of state at adjacent time positions. If the magnitude of change of any state sequence at any time position exceeds the corresponding preset range, it is determined that the state sequence has shifted.

[0116] When the state sequence shifts, the generation process is retried based on the original input sequence, and the set of lens state anchor points and the set of lens generation conditions remain unchanged during the regeneration process until the magnitude of the state change meets the preset range requirements.

[0117] Based on the generated results where the magnitude of state change meets the preset range, the candidate shots that meet the requirements are collected to form a candidate shot set.

[0118] In this embodiment, the improved Sora model is specifically as follows:

[0119] Receive an input sequence consisting of a shot generation intention structure and a shot generation condition set. The input sequence is arranged according to the narrative order index and the time index. Each time index position corresponds to a set of shot state anchor point information and a set of shot generation condition information. The time indices in the input sequence maintain an equal interval order relationship.

[0120] The set of shot generation conditions in the input sequence is split into shot type field, emotion intensity field, narrative order field and subject focus field. The values ​​of each field are converted into continuous numerical representation according to a preset encoding table. The continuous numerical representation is converted into a vector representation of a unified dimension through a linear mapping method. The vectors corresponding to multiple time index positions are arranged in chronological order to form a condition vector sequence. Preferably, the condition vector dimension is set to 128 dimensions.

[0121] The set of shot state anchor points in the input sequence is split into narrative progression state field, emotional state field and visual subject state field. Each state field is mapped to a numerical sequence according to the label order corresponding to the shot generation intention structure. The numerical sequence is compressed to between 0 and 1 through normalization and converted into a vector representation. The vectors corresponding to multiple time index positions are arranged according to the narrative order to form an anchor vector sequence. Preferably, the anchor vector dimension is set to 128 dimensions.

[0122] The condition vector sequence and the anchor vector sequence are aligned in the time index dimension. The alignment is achieved by matching the condition vector and the anchor vector at the same time index position. The aligned vectors are then concatenated in the feature dimension. The concatenation order is fixed as condition vector first and anchor vector second. The concatenation results in a joint control vector sequence, which is used to simultaneously constrain narrative progression, emotional changes, and visual subject focus.

[0123] The timing generation process is driven by a joint control vector sequence. At each time index position, the timing generation process generates a corresponding video frame representation based on the constraint information contained in the joint control vector. The video frames generated at multiple time index positions are arranged in chronological order to form a continuous shot frame sequence. The continuous shot frame sequence is divided into multiple candidate shots according to a preset frame window.

[0124] The image features of each frame are read from a continuous sequence of shot frames. The narrative state features are obtained by statistically analyzing the continuous changes of actions in the image. The emotional state features are obtained by statistically analyzing the color distribution, brightness changes, and character expression changes in the image. The visual subject state features are obtained by statistically analyzing the positional stability and clarity changes of the main object in the image. The features are arranged in chronological order to form the narrative state sequence, emotional state sequence, and visual subject state sequence, respectively.

[0125] In each state sequence, the difference between the state feature values ​​of adjacent time index positions is calculated. The difference calculation is completed by subtracting the previous time index state feature value from the subsequent time index state feature value and taking the absolute value to obtain the corresponding state change amplitude. The state change amplitudes of multiple time index positions are arranged in order to form the state sequence offset.

[0126] The state sequence offset is compared with the preset range item by item. The preset range is used to limit the allowable range of state change at adjacent time index positions. Preferably, the allowable range of narrative state sequence offset is set to 0.3, the allowable range of emotional state sequence offset is set to 0.2, and the allowable range of visual subject state sequence offset is set to 0.25. When the state change amplitude at any time index position exceeds the corresponding allowable range, it is determined that the state sequence offset result exceeds the preset range.

[0127] When the state sequence offset result exceeds the preset range, the vector value at the corresponding position in the joint control vector sequence is adjusted based on the joint control vector at the corresponding time index position in the original input sequence. The adjustment is accomplished by reducing the weight of the change amplitude of the corresponding state field, and the timing generation process is re-triggered to generate a new continuous shot frame sequence.

[0128] When the state sequence offset result is within a preset range, the corresponding continuous shot frame sequence is retained and the candidate shots formed by dividing it are collected and output. Multiple candidate shots that meet the state sequence offset requirements together constitute a candidate shot set.

[0129] In this embodiment, the step of extracting the narrative progression tag set, emotion tag set, and visual attention tag set based on the shot generation intention structure, and constructing the shot state anchor point set, specifically involves:

[0130] The narrative progression tag set is read from the shot generation intention structure. Each narrative progression tag in the narrative progression tag set carries an event sequence identifier. The event sequence identifier is an integer representing the sequential position of the narrative in the overall plot. The value range of the event sequence identifier is consistent with the number of narrative segments. The narrative progression tag set is sorted according to the numerical relationship of the event sequence identifier from small to large. The sorting results are arranged in order to form a narrative progression tag sequence.

[0131] The set of emotion tags is read from the shot generation intention structure. Each emotion tag in the set of emotion tags carries an emotion change sequence identifier. The emotion change sequence identifier describes the stage of emotion change in the development of the plot in integer form. The value range of the emotion change sequence identifier is consistent with the length of the narrative advancement tag sequence. The set of emotion tags is sorted according to the numerical relationship of the emotion change sequence identifier from small to large. The sorting results are arranged in order to form an emotion tag sequence.

[0132] The set of visual attention tags is read from the shot generation intention structure. Each visual attention tag in the set contains a subject identifier and a spatial attention order identifier. The subject identifier is used to distinguish different people or key objects. The spatial attention order identifier describes the switching order of visual focus in the picture in integer form. The value range of the spatial attention order identifier is consistent with the length of the narrative progression tag sequence. The set of visual attention tags is sorted according to the numerical relationship of the spatial attention order identifier from small to large. The sorting results are arranged in order to form a visual attention tag sequence.

[0133] Using the sequence length of the narrative progression tag sequence as the baseline length, the emotion tag sequence and the visual attention tag sequence are indexed and aligned according to the same index position relationship. The index alignment process is completed by dividing the narrative progression tags, emotion tags and visual attention tags with the same index position into the same position. Multiple index positions form multiple aligned tag groups. Preferably, when the length of the emotion tag sequence or the visual attention tag sequence is less than the length of the narrative progression tag sequence, the missing position is filled by copying adjacent tags.

[0134] For each aligned tag group, a combination process is completed. The combination process is completed by aggregating the narrative progression tag, emotion tag, and visual attention tag at the same index position in a fixed order. The aggregation result forms a combined tag unit. The combined tag unit retains the correspondence between narrative order information, emotion change information, and subject attention information.

[0135] Multiple combined tag units are arranged in order of their index positions, and the arrangement order is consistent with the narrative progression tag sequence. The set of combined tag units after arrangement forms a shot state anchor set. The shot state anchor set is used to constrain the narrative continuity, the emotional evolution continuity and the stability of the visual subject in the subsequent shot generation process. Preferably, the index interval between adjacent combined tag units in the shot state anchor set is set to one.

[0136] In this embodiment, S5 specifically refers to:

[0137] The continuous shot frame sequence corresponding to each candidate shot is read one by one from the candidate shot set. The state is analyzed by analyzing the content change relationship between adjacent frames in the continuous shot frame sequence. The narrative state sequence, emotional state sequence and visual subject state sequence are obtained one-to-one with the time sequence. The narrative state sequence is used to describe the unfolding order of plot events in the candidate shot, the emotional state sequence is used to describe the change of the emotion of the picture on the time axis, and the visual subject state sequence is used to describe the attention distribution of the main characters or key objects in the picture in different frames.

[0138] The narrative progression tag set, emotion tag set, and visual attention tag set are read from the shot generation intention structure. The narrative progression tag set and the narrative state sequence maintain the same length correspondence in the time dimension. The emotion tag set and the emotional state sequence maintain the same length correspondence in the time dimension. The visual attention tag set and the visual subject state sequence maintain the same length correspondence in the time dimension.

[0139] The narrative state sequence is mapped one-to-one with the narrative progression tag set according to the index position. For each corresponding position, the event progression direction reflected in the narrative state is compared with the event sequence identifier marked in the narrative progression tag. When the two sequences are consistent, it is recorded as a matching state. When the two sequences are inconsistent, it is recorded as a deviation state. The narrative consistency result is formed by summing the number of matching states and the number of deviation states based on all index positions. Preferably, when the proportion of matching states is not less than 80%, the narrative consistency result is judged to meet the requirements.

[0140] The emotional state sequence is mapped one-to-one with the emotional label set according to the index position. For each corresponding position, the direction of change of emotional intensity reflected in the emotional state is compared with the emotional change sequence marked in the emotional label. When the change direction is consistent, it is recorded as a matching state. When the change direction is inconsistent, it is recorded as a deviation state. The emotional consistency result is formed by summing the number of matching states and the number of deviation states based on all index positions. Preferably, when the proportion of matching states is not less than 75%, the emotional consistency result is judged to meet the requirements.

[0141] The visual subject state sequence is matched one-to-one with the visual attention label set according to the index position. For each corresponding position, the subject distribution reflected in the visual subject state is compared with the subject identifier and spatial attention order identifier marked in the visual attention label. When the subjects are consistent and the spatial attention order is consistent, it is recorded as a matching state. When the subjects or spatial attention order are inconsistent, it is recorded as a deviation state. The attention consistency result is formed by summing the number of matching states and the number of deviation states based on all index positions. Preferably, when the proportion of matching states is not less than 85%, the attention consistency result is judged to meet the requirements.

[0142] The narrative consistency result, emotional consistency result, and attention consistency result are fused according to preset weights. The preset weights are set to 0.4 for narrative consistency result, 0.3 for emotional consistency result, and 0.3 for attention consistency result. The fusion process is completed by weighted summarizing the satisfaction of the three types of consistency results. The fused summarization result forms the shot consistency result.

[0143] Based on the shot consistency results, a shot state description is generated, which includes the narrative consistency indicator, emotional consistency indicator, and attention consistency indicator corresponding to the candidate shot.

[0144] In this embodiment, S6 specifically refers to:

[0145] Each candidate shot is read one by one from the candidate shot set, and a unique shot identifier is generated for each candidate shot. The shot identifier consists of a fixed-length character sequence and includes the sorting position code of the candidate shot in the candidate shot set and the generation batch code. Preferably, the length of the shot identifier is set to 20 characters, where the first 10 characters represent the generation batch code and the last 10 characters represent the sorting position code.

[0146] A lens index table is constructed based on the lens identifier. The lens index table sets a lens identifier field, a lens storage location field, and a lens status description reference field. The lens index table is used to record the logical mapping relationship of candidate lenses in the management unit.

[0147] Shot identifiers and corresponding shot state descriptions are written into the shot index table to form shot registration records. The shot registration records simultaneously retain narrative consistency identifiers, emotional consistency identifiers, and attention consistency identifiers to characterize the comprehensive state features of candidate shots.

[0148] A time stamp is written into the lens registration record. The time stamp is generated using a unified time base and records the time order in which candidate lenses enter the lens management unit in the form of millisecond-level timestamps. Preferably, the time stamp precision is set to 1 millisecond.

[0149] All shot registration records are sorted based on time stamps to form an evolutionary sequence table, which reflects the temporal relationship of candidate shots in the generation and management process.

[0150] The candidate shot content data is associated with the corresponding shot registration record and written into the storage space to form a shot storage record. The shot storage record contains both the data address information and shot identification information of the candidate shot to support subsequent fast retrieval.

[0151] Information is collected based on shot identifiers, time identifiers, shot storage records, and shot status descriptions. Shot evolution records are generated in the order of the evolution sequence table. The shot evolution records describe the generation order, status changes, and storage relationships of candidate shots in the shot management unit in the form of time series, which is used to support the output of film and television shot results and historical tracing.

[0152] In this embodiment, S7 specifically refers to:

[0153] The lens identifier, time identifier, and lens storage record corresponding to each evolution record are read from the lens evolution record. The lens storage record contains the data address information, storage medium identifier, and data integrity verification identifier of the candidate lens. The data integrity verification identifier is generated by combining the candidate lens data length and a fixed verification factor. Preferably, the verification factor is 101.

[0154] The time series of the stored footage is sorted according to the time stamps. The sorting rules are arranged from smallest to largest according to the time stamps to form an output sequence. The output sequence is used to reflect the chronological order of the candidate footage in the process of footage evolution.

[0155] Based on the output sequence, the data address information corresponding to the lens storage record is accessed one by one. Candidate lens content data is read from the storage medium and combined in the order of the output sequence to generate a continuously arranged film and television lens result. The film and television lens result consists of multiple segments of candidate lens content data and maintains temporal continuity.

[0156] The results of the film and television shots are written into the output record. The output record contains the generation time identifier of the film and television shot results, the number of candidate shots and the total data length. Preferably, the generation time identifier is set to a precision of 1 millisecond.

[0157] The output record is associated with the corresponding shot identifier and written, so that the result of each film and television shot can be traced back to the corresponding shot evolution record through the shot identifier;

[0158] The film and television shot results and shot evolution records are jointly stored to generate management storage records. The management storage records simultaneously retain shot identifiers, time identifiers, output sequence order information and film and television shot result storage address information, which are used to support the retrieval, backtracking and continuous management of film and television shot results.

[0159] Example 1:

[0160] To verify the feasibility of this invention in practice, it was applied to three key drama projects of a film and television production team under a provincial radio and television station: urban romance drama A, historical drama B, and suspense drama C. The method of this invention was introduced to assist in the entire process of shot generation and management within three months, with a focus on evaluating indicators such as shot generation efficiency, consistency assessment accuracy, and shot scheduling and management capabilities.

[0161] In series A, the main challenge faced by the creative team was the rapid emotional shifts between multiple characters and the frequent changes in visual style. Traditional manual shot planning methods, while fulfilling the director's intended expression, suffer from significant stylistic inconsistencies and narrative jumps. The method proposed in this invention first collects script fragments, character settings, emotional tone, and visual style elements. Through semantic parsing, it generates a shot generation intention structure. Then, using the constructed set of shot generation conditions, it introduces an improved Sora model to quickly generate a set of candidate shots while ensuring the continuity of narrative, emotion, and subject state. After the system automatically completes shot consistency assessment and management, the recommended shots can be directly used, significantly reducing the workload of storyboard designers.

[0162] The system was applied to three TV series. Throughout the process, each episode's script required approximately 120-160 shots. The system's automatic processing and management ratio is shown in the table below.

[0163] Table 1 Comparison of shot generation efficiency in three TV series

[0164] A 1440 1104 18 76.7% B 1360 998 21 73.4% C 1280 1012 17 79.0%

[0165] As shown in Table 1, the method of this invention achieved over 70% automatic generation of shots in all three dramas, with an average shot generation time of less than 20 minutes per episode. This is significantly better than the traditional manual composition process and greatly improves the efficiency of script-to-film conversion. Among them, C had the highest accuracy rate in automatic generation due to its relatively clear narrative thread.

[0166] Further evaluation was conducted to assess the level of acceptance among the various production crew members regarding the system-generated shots, using two dimensions: consistency score and feedback optimization ratio.

[0167] Table 2 Consistency Assessment and Feedback Optimization Analysis Table

[0168] A 8.2 128 91.4% B 7.9 156 87.2% C 8.5 96 93.8%

[0169] As shown in Table 2, the consistency scores for all three series are above 7.9, indicating that the improved Sora model has high stability in shot state anchor construction and state conservation verification. Particularly in C, only 96 shots were adjusted based on feedback, resulting in a final retention rate of 93.8%, demonstrating that the system-generated shots closely match the director's intentions.

[0170] Analysis of the two data tables above reveals that this invention not only significantly improves the efficiency of automatic shot generation but also offers advantages in shot quality control and consistency assurance. The improved Sora model, through deep integration of the shot generation intention structure and condition set, achieves dynamic consistency between visual subject focus, emotional tone, and narrative rhythm. The shot management unit further enhances the traceability of shot evolution recording, effectively solving problems such as fragmentation, jumps, and incoherence in traditional manual shot planning. This provides a practical and intelligent solution for creation and management in the film and television industrialization process.

[0171] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for generating and managing film and television shots based on generative artificial intelligence, characterized in that, Includes the following steps: S1. Collect input information for film and television creation and construct a set of shot generation requirements; specifically: Collect script excerpts, character settings, scene settings, emotional tone, and visual style information; The script fragment information is processed by sentence segmentation to obtain a set of narrative fragments; the character setting information is processed by entity normalization to obtain a set of character entities; the scene setting information is processed by scene tagging to obtain a set of scene tags; the emotional tone information is processed by emotional tagging to obtain a set of emotional tags; and the visual style information is processed by style tagging to obtain a set of style tags. Based on the set of narrative fragments, character entities, scene tags, emotion tags, and style tags, a structured splicing process is performed to construct a set of shot generation requirements descriptions; S2. Perform semantic parsing processing on the shot generation requirement description set to generate a shot generation intent structure; specifically: The shot generation requirement description set is processed by word segmentation and syntactic analysis to obtain a sequence of semantic units and a sequence of dependency relations. Based on the semantic unit sequence, event element extraction processing is performed to obtain the action word set, subject word set, object word set and modifier word set; Based on the dependency relation sequence, the action word set, subject word set, object word set, and modifier set are combined to obtain the narrative event set; Perform shot intent tag mapping on the narrative event set to generate a shot function tag set, an emotion tag set, a narrative progression tag set, and a visual attention tag set; Based on the set of shot function labels, set of emotion labels, set of narrative progression labels, and set of visual attention labels, vectorization processing is performed and the data is spliced ​​to generate a shot generation intention structure. S3. Construct a set of shot generation conditions based on the shot generation intention structure; Specifically: Read the set of shot function tags, set of emotion tags, set of narrative progression tags, and set of visual attention tags in the shot generation intention structure; Perform shot type mapping processing on the shot function tag set to generate shot type conditions; perform emotion intensity calibration processing on the emotion tag set to generate emotion conditions; perform event sequence calibration processing on the narrative progression tag set to generate narrative conditions; perform subject focus calibration processing on the visual attention tag set to generate attention conditions. The conditions for shot type, emotion, narrative, and attention are spliced ​​together to construct a set of shot generation conditions. S4. Input the shot generation intention structure and the shot generation condition set into the improved Sora model. The improved Sora model constructs shot state anchor points and introduces a state conservation verification mechanism to generate a candidate shot set; specifically: Read the shot generation intention structure and shot generation condition set; Based on the shot generation intention structure, extract the narrative progression tag set, emotion tag set, and visual attention tag set, and construct the shot state anchor point set; The set of shot state anchor points and the set of shot generation conditions are combined to generate an input sequence, and the input sequence is fed into the improved Sora model. Candidate shots are generated based on the improved Sora model, and state extraction processing is performed on the candidate shots to obtain narrative state sequence, emotional state sequence and visual subject state sequence. State conservation verification is performed on the narrative state sequence, emotional state sequence and visual subject state sequence. If the offset of any state sequence is greater than the preset range, regeneration is performed based on the input sequence. The output of the state conservation verification process is used to collect the generated candidate shots and form a candidate shot set. The improved Sora model is specifically as follows: Receive an input sequence consisting of a shot generation intention structure and a shot generation condition set, wherein the input sequence is arranged in chronological order; The condition set of shots in the input sequence is generated by performing vector mapping processing to obtain a condition vector sequence; Perform state encoding processing on the set of shot state anchor points in the input sequence to obtain an anchor point vector sequence; The condition vector sequence and the anchor vector sequence are aligned, and the aligned vectors are concatenated to form a joint control vector sequence. Temporal reasoning is performed based on the joint control vector sequence to generate a continuous shot frame sequence, which constitutes a candidate shot. Feature extraction processing is performed on a continuous sequence of shot frames to obtain a narrative state sequence, an emotional state sequence, and a visual subject state sequence. The state sequence offset is obtained by calculating the state difference between adjacent frames for the narrative state sequence, the emotional state sequence, and the visual subject state sequence. The state sequence offset is compared with the preset range to obtain the state sequence offset result; If the state sequence offset result exceeds the preset range, the joint control vector sequence is adjusted based on the input sequence and the temporal inference generation is completed again; If the state sequence offset result is within the preset range, then output a set of candidate shots; The process of extracting narrative progression tag sets, emotion tag sets, and visual attention tag sets based on the shot generation intention structure to construct a shot state anchor point set is as follows: Extract the corresponding event sequence identifier for each narrative advancement tag in the narrative advancement tag set, and sort the narrative advancement tag set according to the event sequence identifier to generate a narrative advancement tag sequence; Extract the corresponding emotion change sequence identifier for each emotion tag in the emotion tag set, and sort the emotion tag set according to the emotion change sequence identifier to generate an emotion tag sequence. For each visual attention tag in the visual attention tag set, extract the corresponding subject identifier and spatial attention order identifier, and sort the visual attention tag set according to the spatial attention order identifier to generate a visual attention tag sequence. Based on the sequence length of the narrative progression tag sequence, the emotion tag sequence and the visual attention tag sequence are index aligned according to their index positions to generate multiple aligned tag groups; Perform a combination process on each aligned tag group to form a combined tag unit that includes narrative progression tags, emotion tags, and visual attention tags; Arrange multiple combined label units in sequence to generate a set of lens state anchor points; S5. Based on the shot generation intent structure, perform a consistency evaluation process on the candidate shot set to generate a shot state description; specifically: For each candidate shot in the candidate shot set, extract the corresponding narrative state sequence, emotional state sequence, and visual subject state sequence; Extract the set of narrative progression tags, the set of emotion tags, and the set of visual attention tags from the shot generation intention structure; Based on the narrative advancement tag set, state matching processing is performed on the narrative state sequence to obtain narrative consistency results; Based on the set of emotion labels, state matching processing is performed on the emotion state sequence to obtain emotion consistency results; State matching processing is performed on the visual subject state sequence based on the visual attention tag set to obtain attention consistency results; The narrative consistency result, emotional consistency result, and attention consistency result are fused to generate the shot consistency result; a shot state description is generated based on the shot consistency result. S6. Construct a shot management unit based on the shot state description, and input the candidate shots in the candidate shot set and their corresponding shot state descriptions into the shot management unit to generate a shot evolution record; specifically: Assign a lens identifier to each candidate lens in the candidate lens set, and build a lens index table based on the lens identifier; Write the lens identifier and corresponding lens status description into the lens index table to generate a lens registration record; write a time identifier into the lens registration record and establish an evolution sequence table based on the time identifier; The candidate shots are associated with the shot registration records and stored to generate shot storage records; A lens evolution record is generated based on lens identifiers, time identifiers, lens storage records, and lens state descriptions; S7. Based on the shot evolution record, output the film and television shot results from the shot management unit, and associate and store the film and television shot results with the corresponding shot evolution record to realize the generation and management of film and television shots.

2. The method for generating and managing film and television shots based on generative artificial intelligence according to claim 1, characterized in that, Specifically, S7 is: Lens identifiers, time identifiers, and lens storage records are extracted from the lens evolution record; the lens storage records are sorted according to the time identifiers to generate an output sequence; Candidate shots are extracted from the shot storage record based on the output sequence to generate film and television shot results; Write the video shot results into the output record and associate the output record with the shot identifier; The film and television shot results are associated with the shot evolution record and stored together to generate management storage records.

Images