Editable video generation and local reconstruction based on structured intermediate states

Abstract

The application discloses an editable video generation and local reconstruction method based on a structured intermediate state, provides a visual editing interface, generates an incremental reconstruction signal in response to a local modification operation performed by a user on target shot data in the visual editing interface, locates a data unit to be reconstructed in an initial structured video intermediate state according to a unique shot identifier in the incremental reconstruction signal and a path identifier of the modified data, only inputs the data unit to be reconstructed and adjacent connection data units of the data unit to be reconstructed into a video intermediate state generation intelligent agent for local regeneration, obtains a target structured video intermediate state, and generates a target complete preview video by performing incremental splicing and synthesis from a position where the target shot data is located according to the target structured video intermediate state, and displays the target complete preview video in the visual editing interface. The problems that the prior art cannot realize local regeneration of only a single shot and keeping unchanged of splicing results before the target shot are solved.

Description

Technical Field

[0001] This invention relates to the field of digital media technology, and more specifically, to a method for generating and partially reconstructing editable videos based on structured intermediate states. Background Technology

[0002] The short video industry is currently experiencing explosive growth, and the market demand for efficient video generation tools is increasing daily.

[0003] The prior art (application number: 202511044744.8) discloses an AI intelligent short video generation method based on multi-agent collaboration. This method edits and synthesizes multiple storyboard video clips into a complete video according to the script order. If the user needs to modify a certain storyboard, the method does not have local regeneration. All modifications require re-execution of the material acquisition and storyboard video generation steps, regenerating all storyboard video clips and re-executing the entire editing and synthesis process. It cannot achieve local regeneration of only a single storyboard and keep the splicing results of the target storyboard unchanged. Therefore, it is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0004] In view of this, the present invention provides an editable video generation and local reconstruction method based on structured intermediate states, which solves the problem that the existing technology does not have local regeneration, and all modifications require re-execution of the material acquisition and shot video generation steps, regeneration of all shot video segments and re-execution of the entire editing and compositing process, and cannot achieve local regeneration of only a single shot and keep the splicing results of the target shot unchanged.

[0005] This application provides a method for generating and partially reconstructing editable videos based on structured intermediate states, including the following steps:

[0006] The system obtains the video script creation requirements input by the user and inputs these requirements into the video intermediate state generation agent. The multimodal feature encoding network of the video intermediate state generation agent performs multidimensional feature analysis on the video script creation requirements in terms of text, style, and timing to generate an initial structured video intermediate state. The initial structured video intermediate state uses storyboard data as the basic storage unit. Each storyboard data includes: a unique storyboard identifier, storyboard duration data, content description dataset, visual material metadata, and audio material metadata.

[0007] A visual editing interface is provided, which is used to present the content description dataset, visual material metadata and audio material metadata of each scene data in the initial structured video intermediate state, and supports users to perform local modification operations on any data in any scene data;

[0008] In response to the user's partial modification operation on the target data or target material metadata of the target storyboard data in the visual editing interface, an incremental reconstruction signal is generated, and a mapping relationship is established between each of the target storyboard data or the target material metadata and the storyboard data; the incremental reconstruction signal includes the unique storyboard identifier of the target storyboard data and the path identifier of the modified data;

[0009] Based on the unique scene identifier and the path identifier of the modified data in the incremental reconstruction signal, the data unit to be reconstructed is located in the initial structured video intermediate state; only the data unit to be reconstructed and its adjacent connecting data units are input to the video intermediate state generation agent for local regeneration to obtain the target structured video intermediate state; all data of the scene data in the target structured video intermediate state that has not undergone the local modification operation remains unchanged;

[0010] Based on the intermediate state of the target structured video, incremental stitching and synthesis are performed starting from the location of the target storyboard data to generate a complete preview video of the target, and the complete preview video of the target is displayed in the visual editing interface.

[0011] Optionally, the visual material metadata includes index information of the frame sequence, style feature vectors, and inter-frame optical flow field information, and the audio material metadata includes temporal parameters of the audio waveform and timbre feature vector data;

[0012] The adjacent connection data unit includes the forward tail frame optical flow field information and the backward first frame optical flow field information of the target segment data; during the local regeneration, the video intermediate state generation agent uses the forward tail frame optical flow field information and the backward first frame optical flow field information as boundary conditions to ensure that the target segment data and the forward tail frame optical flow field information and the backward first frame optical flow field information in the target structured video intermediate state maintain a smooth transition of optical flow field in terms of image motion continuity.

[0013] Optionally, the recomputation cost of the local regeneration must satisfy the following expression:

[0014] ,

[0015] in, This represents the amount of recomputation required for local regeneration. The segment duration data refers to the target segment data. The computational resource consumption coefficient for generating visual material per unit duration for the intelligent agent in the video intermediate state. The audio duration data corresponding to the audio material metadata of the target storyboard data. The computational resource consumption coefficient for generating audio material per unit duration for the intelligent agent generating the intermediate state of the video. This indicates that visual regeneration is enabled. This indicates that audio regeneration is enabled; its value can be 0 or 1. The recalculation amount of the local regeneration increases linearly with the segment duration data of the target segment data, and is independent of the total recalculation amount of all segment data in the initial structured video intermediate state.

[0016] Optionally, the total weight calculation of all split-cam data in the initial structured video intermediate state is expressed as:

[0017] ,

[0018] In the formula, N is the total number of all split-cam data. Indicates the first The segment duration data of each segment data. Indicates the first The audio duration data corresponding to the audio material metadata of each storyboard data.

[0019] Optionally, the incremental stitching synthesis includes: keeping the stitching results of all confirmed storyboard data before the target storyboard data unchanged, starting from the target storyboard data, re-stitching the visual material metadata and audio material metadata of the target storyboard data and its subsequent storyboard data in sequence; the playback timeline of the target complete preview video is updated after the start timestamp of the target storyboard data, while the playback timeline before the target storyboard data remains unchanged.

[0020] Compared with existing technologies, the editable video generation and local reconstruction method based on structured intermediate states provided by this invention achieves at least the following beneficial effects:

[0021] First, by constructing a data storage and addressing system using a unique segment identifier as an index in the initial structured video intermediate state, and combining it with the path identifier of the modified data carried by the incremental reconstruction signal, as well as the scope definition of local regeneration, a precise mapping from local modification operations to the data units to be reconstructed in the target segment data is achieved. This ensures that any local modification operation only affects the record of the target segment data corresponding to the unique segment identifier, while the data of other segment data remains isolated at the physical storage level. This fundamentally reduces the modification cost of the traditional "full regeneration" to the local cost of "only recalculating the data units to be reconstructed and adjacent connected data units".

[0022] Second, by using adjacent connecting data units in local regeneration as boundary conditions to participate in regeneration, and in conjunction with incremental splicing synthesis starting from the location of the target storyboard data, the modified video maintains a natural connection with the unmodified part in terms of visual style, motion continuity and playback timeline, while eliminating the computational overhead of resynthesizing the entire video from scratch.

[0023] Of course, any product implementing the present invention does not need to achieve all of the above additional technical effects while solving the background technical problem. It is sufficient for the product to solve the background technical problem first. The additional technical effects are effects that are beyond the understanding of those skilled in the art when the specific structure of the present invention is combined with a specific environment.

[0024] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0026] Figure 1 This is a flowchart illustrating the editable video generation and local reconstruction method based on structured intermediate states provided by the present invention. Detailed Implementation

[0027] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0028] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0029] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0030] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0031] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0032] See Figure 1 As shown, Figure 1 This is a flowchart illustrating the editable video generation and local reconstruction method based on structured intermediate states provided by the present invention. This embodiment provides an editable video generation and local reconstruction method based on structured intermediate states, characterized by including the following steps:

[0033] Step S1: Obtain the video script creation requirements input by the user, input the video script creation requirements into the video intermediate state generation agent, and perform multi-dimensional feature analysis of the video script creation requirements in terms of text, style, and time sequence through the multimodal feature encoding network of the video intermediate state generation agent to generate an initial structured video intermediate state; the initial structured video intermediate state uses the storyboard data as the basic storage unit, and each storyboard data includes: a unique storyboard identifier, storyboard duration data, content description dataset, visual material metadata, and audio material metadata;

[0034] Specifically, the aforementioned video intermediate state generation agent is implemented through a large language model, which can be either the Doubao model or the DeepSeek model; this embodiment does not limit this to any particular model.

[0035] The visual material metadata is the metadata of the generated visual materials, and the audio material metadata is the metadata of the generated audio materials.

[0036] The aforementioned initial structured video intermediate state transforms user requirements into structured data records with storyboard data as the granularity and visual and audio material metadata as the content, rather than directly outputting the final video file.

[0037] Optionally, the visual material metadata includes index information and style feature vectors of the image frame sequence, and the audio material metadata includes temporal parameters of the audio waveform and timbre feature vector data.

[0038] By storing visual and audio metadata in the initial structured video intermediate state using a unique storyboard identifier as an index, any subsequent local modification operation can be accurately located in the initial structured video intermediate state based on the unique storyboard identifier, thus achieving data read / write isolation with storyboard data as the smallest granularity.

[0039] Step S2: Provide a visual editing interface. The visual editing interface is used to present the content description dataset, visual material metadata and audio material metadata of each scene data in the initial structured video intermediate state, and supports users to perform local modification operations on any data of any scene data.

[0040] Users can directly view and modify any data unit in the initial structured video intermediate state through a visual editing interface, such as unique storyboard identifiers, storyboard duration data, content description datasets, visual material metadata, and audio material metadata; without having to manipulate the original video file or re-enter creative requirements.

[0041] The aforementioned partial modification operations include data value editing, material metadata replacement, and timing adjustment;

[0042] Step S3: In response to the user's partial modification operation on the target data or target material metadata of the target storyboard data in the visual editing interface, an incremental reconstruction signal is generated, and a mapping relationship is established between each target storyboard data or target material metadata and the storyboard data; the incremental reconstruction signal contains the unique storyboard identifier of the target storyboard data and the path identifier of the modified data;

[0043] A one-to-one mapping relationship is established between the storyboard data of each target and the storyboard data.

[0044] The incremental reconstruction signal also carries the modified content of the target storyboard data; the system uses the incremental reconstruction signal to accurately locate the location and content of the user's modifications, rather than recording the modifications as a whole.

[0045] By using the incremental reconstruction signal in step S3 to carry the path identifier of the modified data, the user's modification action is quantified into a parsable structured instruction, thus achieving a precise mapping from local modification operations to the corresponding data units in the initial structured video intermediate state.

[0046] Step S4: Based on the unique storyboard identifier and the path identifier of the modified data in the incremental reconstruction signal, locate the data unit to be reconstructed in the initial structured video intermediate state; input only the data unit to be reconstructed and its adjacent connected data units to the video intermediate state generation agent for local regeneration to obtain the target structured video intermediate state; all data of the storyboard data in the target structured video intermediate state that have not undergone local modification operation remain unchanged.

[0047] In step S4, by using the data unit to be reconstructed and its adjacent connecting data units as inputs for local regeneration, the target segment data and adjacent segment data in the intermediate state of the target structured video maintain a smooth transition in visual style and motion continuity.

[0048] The system only sends the data unit to be reconstructed and its adjacent connected data units into the video intermediate state to generate the intelligent agent, without involving the data of the unmodified storyboard data.

[0049] In step S1, the initial structured video intermediate state uses storyboard data as the basic storage unit, with each storyboard data corresponding to a unique storyboard identifier. In step S3, the incremental reconstruction signal carries the unique storyboard identifier of the target storyboard data and the path identifier of the modified data. In step S4, the data unit to be reconstructed is located in the initial structured video intermediate state based on the unique storyboard identifier and the path identifier of the modified data in the incremental reconstruction signal. This scheme ensures that any local modification operation only affects the record of the target storyboard data corresponding to the unique storyboard identifier, while the data of other storyboard data remains isolated at the physical storage level and is not accessed.

[0050] In step S3, the incremental reconstruction signal carries a unique storyboard identifier for the target storyboard data, a path identifier for the modified data, and the content of the modification. In step S4, the data unit to be reconstructed is located in the initial structured video intermediate state based on the unique storyboard identifier and the path identifier for the modified data in the incremental reconstruction signal. This scheme enables the system to identify the precise data location corresponding to each local modification operation, providing a precise range definition for subsequent local regeneration.

[0051] In step S4, only the data unit to be reconstructed and its adjacent connecting data units are input into the video intermediate state generation agent for local regeneration. All data in the target structured video intermediate state that has not undergone local modification remains unchanged. This scheme uses adjacent connecting data units as boundary conditions to participate in regeneration, so that the updated target segment data can be naturally connected with the visual style and motion trajectory of the adjacent segment data during stitching.

[0052] Optionally, in step S4, the amount of recalculation for local regeneration... The following expression must be satisfied:

[0053] ,

[0054] in, This represents the amount of recomputation required for local regeneration. The storyboard duration data is for the target storyboard data. The computational resource consumption coefficient for generating visual footage per unit duration for the intelligent agent that generates video intermediate states. The audio duration data corresponding to the audio material metadata of the target storyboard data. The computational resource consumption coefficient for generating audio material per unit duration for the intelligent agent that generates video intermediate states. and These are the visual regeneration enable flag and the audio regeneration enable flag, respectively, with values ​​of 0 or 1. The amount of recomputation required for local regeneration The segment duration data increases linearly with the target segment data, and the total weight calculation is compared with the total weight calculation of all segment data in the initial structured video intermediate state. Irrelevant, where N is the total number of all split-cam data. Indicates the first Storyboard duration data for each storyboard. Indicates the first The audio duration data corresponding to the audio material metadata of each storyboard data.

[0055] The expression for the recomputation cost of local regeneration limits the computational overhead of local regeneration triggered by a single modification to depend only on the segment duration and audio duration data of the target segment data itself, and is independent of the total number of segment data N. This fundamentally reduces the computational complexity of a single modification from Down to Specifically, this involves reducing the recomputation cost of locally regenerated data. The formula constraint allows the computational cost of local regeneration to be quantified solely by the segment duration data of the target segment data. Audio duration data corresponding to the audio material metadata of the target storyboard data Visual regeneration activation mark and audio regeneration enable flag The determined linear function has a computational complexity that is lower than traditional Down to O(1), the computational cost of a single modification does not increase with the total number of all split-scene data N in the initial structured video intermediate state.

[0056] Recalculation cost through local regeneration The total computational cost compared to all split-scene data in the initial structured video intermediate state Irrelevant limitations, and and The visual regeneration enable flag and audio regeneration enable flag are controlled independently, further reducing the actual computational cost of a single modification to the cost of recalculating only the material of the modified modality.

[0057] Visual Regeneration Enabled and audio regeneration enable flag Each value can be independently either 0 or 1, and the sum of the two values ​​must be greater than or equal to 1. When the user only modifies the visual material metadata, , The amount of recomputation required for local regeneration This does not include any audio recalculation costs; when the user only modifies the audio material metadata, , The amount of recomputation required for local regeneration This approach eliminates any visual recalculation costs. It ensures that the computational overhead of a single modification precisely matches the actual requirements of the modified modality, eliminating redundant computations caused by modal coupling.

[0058] Step S5: Based on the intermediate state of the target structured video, perform incremental stitching and synthesis starting from the location of the target storyboard data to generate a complete preview video of the target, and display the complete preview video of the target in the visual editing interface.

[0059] The stitching starts from the location of the target storyboard data rather than from the beginning. The previous storyboard stitching results remain unchanged, and users can see the effect of the changes immediately.

[0060] In step S5, based on the intermediate state of the target structured video, incremental stitching and synthesis are performed starting from the location of the target storyboard data to generate a complete preview video of the target. The stitching results before the target storyboard data remain unchanged, eliminating the need for re-synthesis and removing the computational overhead of re-stitching the entire video due to a single modification.

[0061] In step S2, the visual editing interface presents the content description dataset, visual material metadata, and audio material metadata of each scene data in the initial structured video intermediate state, and supports users to perform local modifications on any data in any scene data or any material metadata. Step S3 generates an incremental reconstruction signal in response to any local modification operation, step S4 performs local regeneration, and step S5 performs incremental stitching and compositing and refreshes the visual editing interface. This solution realizes a complete closed loop of "presentation-modification-reconstruction-refresh," allowing users to see the updated preview video immediately after each modification.

[0062] Optionally, in step S5, incremental splicing and compositing includes: keeping the splicing results of all confirmed segment data before the target segment data unchanged, starting from the target segment data, re-sponsoring the visual material metadata and audio material metadata of the target segment data and its subsequent segment data in order; the playback timeline of the target complete preview video is updated after the start timestamp of the target segment data, while the playback timeline before the target segment data remains unchanged.

[0063] The playback timeline of the target full preview video is updated after the start timestamp of the target storyboard data, while the playback timeline before the target storyboard data remains unchanged.

[0064] By keeping the stitching results of all previously confirmed storyboard data unchanged, the computational scope of incremental stitching is reduced from "stitching all N storyboard data" to "stitching only subsequent storyboard data starting from the target storyboard data," thus reducing the computational cost from O(N) to [missing information]. Where k is the position number of the target storyboard data in the entire storyboard sequence. Specifically, the incremental stitching and compositing in step S5 starts from the target storyboard data. The stitching results of all confirmed storyboard data before the target storyboard data remain unchanged and do not need to be recombined. When the user modifies the k-th storyboard data, it is only necessary to re-stitch from the k-th storyboard data, and the stitching results of the previous k-1 storyboard data are completely preserved. This scheme decouples the stitching and compositing range from the total number of storyboard data N, and the stitching computation depends only on the number of storyboards from the target storyboard data to the end.

[0065] The playback timeline of the target full preview video is updated only after the start timestamp of the target scene data. This ensures that the playback timeline before the target scene data maintains its absolute position on the timeline, eliminating the overhead of recalculating the entire timeline due to partial modifications. Specifically, in step S5, the playback timeline before the target scene data remains unchanged, and the start and end timestamps of each scene data remain consistent before and after the modification. A user's modification to the k-th scene data will not affect the time position of the 1st to k-1th scene data on the playback timeline; only the accumulated timestamps of all scene data starting from the k-th scene data need to be recalculated. This scheme limits the timeline update scope of a single modification to the target scene data and its subsequent parts, avoiding the recalculation of the entire timeline.

[0066] In step S4, local regeneration only inputs the data units to be reconstructed and their adjacent connected data units into the video intermediate state generation agent. Combined with step S5, incremental stitching synthesis reassembles the data sequentially starting from the target storyboard data. This ensures that the visual and audio metadata are seamlessly embedded into the original stitching sequence after regeneration. The regeneration and stitching synthesis stages maintain spatial consistency and are seamlessly connected. Specifically, step S4's local regeneration operates on the target storyboard data and its adjacent connected data units. Step S5's incremental stitching synthesis reassembles the visual and audio metadata of the target storyboard data and its subsequent storyboard data, starting from the target storyboard data. Both steps S4 and S5 use the target storyboard data as the starting point and a unique storyboard identifier as the positioning basis. The updated material generated by local regeneration and the updated intermediate state read by incremental stitching synthesis are completely matched spatially, eliminating data misalignment or duplicate stitching problems caused by inconsistencies between the recalculation and stitching ranges.

[0067] Optionally, in step S1, the visual material metadata includes inter-frame optical flow field information of the frame sequence; in step S4, the adjacent connection data unit includes forward tail frame optical flow field information and backward first frame optical flow field information of the target segment data; during local regeneration, the video intermediate state generating agent uses the forward tail frame optical flow field information and backward first frame optical flow field information as boundary conditions to ensure that the target segment data in the target structured video intermediate state maintains a smooth transition of optical flow field with the forward adjacent segment data and the backward adjacent segment data in terms of the continuity of screen motion.

[0068] Specifically, in step S1, in addition to the index information and style feature vectors of the image frame sequence, the visual material metadata further includes the inter-frame optical flow field information of the image frame sequence. The inter-frame optical flow field information records the motion vectors of each pixel between adjacent frames, which is used to describe the motion direction and speed of objects within the image.

[0069] In step S4, the adjacent connection data unit specifically includes the optical flow field information of the forward tail frame and the backward first frame of the target segment data. During local regeneration, the video intermediate state generation agent uses the optical flow field information of the forward tail frame and the backward first frame as boundary conditions to ensure that the target segment data in the target structured video intermediate state maintains a smooth transition of optical flow field with the forward adjacent segment data and the backward adjacent segment data in terms of the continuity of image motion.

[0070] By adopting the above scheme, by further including the inter-frame optical flow field information of the frame sequence in the visual material metadata in step S1, the motion vector data between adjacent frames within each segment data is saved when the initial structured video intermediate state is generated for the first time, providing a searchable and referable data foundation for motion continuity boundary matching during subsequent local regeneration.

[0071] In step S4, the agent generated by the video intermediate state performs local regeneration using the optical flow field information of the preceding tail frame and the optical flow field information of the following first frame as boundary conditions. This makes the motion vector of the first frame of the target segment data in the target structured video intermediate state continuous with the motion vector of the tail frame of the preceding adjacent segment data, and the motion vector of the tail frame of the target segment data continuous with the motion vector of the first frame of the following adjacent segment data.

[0072] By performing local regeneration using the optical flow field information of the preceding tail frame and the optical flow field information of the following first frame as boundary conditions in step S4, and simultaneously cooperating with the incremental stitching synthesis starting from the location of the target storyboard data in step S5, the intermediate state of the target structured video achieves seamless inter-frame connection in terms of motion continuity, and this connection effect does not depend on the post-interpolation compensation in the stitching synthesis stage.

[0073] Compared with existing technologies, the editable video generation and local reconstruction method based on structured intermediate states provided in this embodiment achieves at least the following beneficial effects:

[0074] First, by constructing a data storage and addressing system using a unique storyboard identifier as an index in the initial structured video intermediate state, and combining this with the path identifier of the modified data carried by the incremental reconstruction signal, as well as the scope definition of local regeneration, a precise mapping from local modification operations to the data units to be reconstructed in the target storyboard data is achieved. This ensures that any local modification operation only affects the record of the target storyboard data corresponding to the unique storyboard identifier, while the data of other storyboards remains isolated at the physical storage level. This fundamentally reduces the modification cost of the traditional "full regeneration" to the local cost of "only recalculating the data units to be reconstructed and adjacent connected data units." Specifically, through the coordinated operation of steps S1, S3, and S4, the system can accurately map any local modification operation by the user to the corresponding data position in the initial structured video intermediate state, and use this position as a benchmark to define the minimum recalculation range, achieving precise end-to-end control from modification instructions to data positioning and then to minimizing recalculation.

[0075] Second, by using adjacent connecting data units as boundary conditions in local regeneration, and in conjunction with incremental stitching synthesis starting from the location of the target storyboard data, the modified video maintains a natural connection with the unmodified part in terms of visual style, motion continuity, and playback timeline, while eliminating the computational overhead of re-synthesizing the entire video from scratch. Specifically, through the coordinated cooperation between steps S4, the visual style and motion trajectory of the regenerated target storyboard data are naturally connected with those of the adjacent storyboard data during stitching, while only updating the playback timeline from the starting position of the target storyboard data and its subsequent parts, eliminating the computational overhead of re-stitching the entire video due to a single modification.

[0076] Third, through bidirectional mapping and closed-loop linkage between the visual editing interface and the initial structured video intermediate state and incremental reconstruction signals, an end-to-end closed loop of "presentation-modification-reconstruction-refresh" is achieved, transforming video creation from linear generation to an iterative creation process that allows for arbitrary backtracking, modification, and real-time feedback. Specifically, steps S2 to S5 form a complete closed-loop link: any modification to the user interface triggers signal generation, the signal drives data positioning, positioning drives local recalculation, recalculation drives incremental stitching, incremental stitching drives interface refresh, and after the interface refresh, the user can immediately make the next round of modifications. This closed-loop link transforms video creation from a one-time linear generation to an iterative creation process that allows for arbitrary backtracking, modification, and real-time feedback.

[0077] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalent elements of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0078] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating and locally reconstructing editable videos based on structured intermediate states, characterized in that, Includes the following steps: The system obtains the video script creation requirements input by the user and inputs these requirements into the video intermediate state generation agent. The multimodal feature encoding network of the video intermediate state generation agent performs multidimensional feature analysis on the video script creation requirements in terms of text, style, and timing to generate an initial structured video intermediate state. The initial structured video intermediate state uses storyboard data as the basic storage unit. Each storyboard data includes: a unique storyboard identifier, storyboard duration data, content description dataset, visual material metadata, and audio material metadata. A visual editing interface is provided, which is used to present the content description dataset, visual material metadata and audio material metadata of each scene data in the initial structured video intermediate state, and supports users to perform local modification operations on any data in any scene data; In response to the user's partial modification operation on the target data or target material metadata of the target storyboard data in the visual editing interface, an incremental reconstruction signal is generated, and a mapping relationship is established between each of the target storyboard data or the target material metadata and the storyboard data; the incremental reconstruction signal includes the unique storyboard identifier of the target storyboard data and the path identifier of the modified data; Based on the unique scene identifier and the path identifier of the modified data in the incremental reconstruction signal, the data unit to be reconstructed is located in the initial structured video intermediate state; only the data unit to be reconstructed and its adjacent connecting data units are input to the video intermediate state generation agent for local regeneration to obtain the target structured video intermediate state; all data of the scene data in the target structured video intermediate state that has not undergone the local modification operation remains unchanged; Based on the intermediate state of the target structured video, incremental stitching and synthesis are performed starting from the location of the target storyboard data to generate a complete preview video of the target, and the complete preview video of the target is displayed in the visual editing interface.

2. The method for generating and locally reconstructing editable video based on structured intermediate states according to claim 1, characterized in that, The visual material metadata includes index information of the frame sequence, style feature vectors, and inter-frame optical flow field information; the audio material metadata includes temporal parameters of the audio waveform and timbre feature vector data. The adjacent connection data unit includes the forward tail frame optical flow field information and the backward first frame optical flow field information of the target segment data; during the local regeneration, the video intermediate state generation agent uses the forward tail frame optical flow field information and the backward first frame optical flow field information as boundary conditions to ensure that the target segment data and the forward tail frame optical flow field information and the backward first frame optical flow field information in the target structured video intermediate state maintain a smooth transition of optical flow field in terms of image motion continuity.

3. The editable video generation and local reconstruction method based on structured intermediate states according to claim 1, characterized in that, The recomputation cost of the local regeneration must satisfy the following expression: ， in, This represents the amount of recomputation required for local regeneration. The segment duration data refers to the target segment data. The computational resource consumption coefficient for generating visual material per unit duration for the intelligent agent in the video intermediate state. The audio duration data corresponding to the audio material metadata of the target storyboard data. The computational resource consumption coefficient for generating audio material per unit duration for the intelligent agent generating the intermediate state of the video. This indicates that visual regeneration is enabled. This indicates that audio regeneration is enabled; its value can be 0 or 1. The recalculation cost of the local regeneration increases linearly with the segment duration data of the target segment data, and is independent of the total recalculation cost of all segment data in the initial structured video intermediate state.

4. The editable video generation and local reconstruction method based on structured intermediate states according to claim 3, characterized in that, The total weight calculation of all split-scene data in the initial structured video intermediate state is expressed as: ， In the formula, N is the total number of all split-cam data. Indicates the first The segment duration data of each segment data. Indicates the first The audio duration data corresponding to the audio material metadata of each storyboard data.

5. The editable video generation and local reconstruction method based on structured intermediate states according to claim 1, characterized in that, The incremental splicing synthesis includes: keeping the splicing results of all confirmed segment data before the target segment data unchanged; starting from the target segment data, re-sponsoring the visual material metadata and audio material metadata of the target segment data and its subsequent segment data in sequence; the playback timeline of the target complete preview video is updated after the start timestamp of the target segment data, while the playback timeline before the target segment data remains unchanged.

Images