A short play generation method and system based on multi-view role anchor points and shot chain feedback

By using a multi-view character anchor point and camera chain feedback method, the problem of character identity feature drift and state discontinuity in short drama generation is solved, achieving consistency of character identity and continuity of state across shots, and reducing computational resource consumption.

CN123123677APending Publication Date: 2026-11-17HANGZHOU YIEN DIGITAL TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611627597.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-21
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing short drama generation technologies struggle to maintain consistency in character identity characteristics under different perspectives, poses, and occlusion conditions. The transitions between adjacent shots are discontinuous, the sources of anomalies are difficult to identify, and the repeated generation of entire shots leads to a waste of computational resources.

Method used

By establishing a multi-perspective character anchor point and shot chain feedback method, the short drama script is analyzed to form a shot chain, a regional perspective baseline map is generated and the coordinate alignment is standardized, a character regional-condition source map is generated, and character features are injected using identity attention branch and temporal control branch, identity and boundary residuals are calculated, local repair is performed and the terminal status is fed back.

Benefits of technology

It improves the consistency of character identity and the continuity of movement across shots, reduces the scope of repeated generation and the consumption of computing resources, and reduces the computing resources and processing time of video generation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123677A_ABST
    Figure CN123123677A_ABST
Patent Text Reader

Abstract

The application discloses a short play generation method and system based on multi-view role anchor points and shot chain feedback. The method analyzes a short play script and establishes a shot chain with state-dependent edges. Multi-view reference maps are partitioned and features are extracted to establish a role anchor point set. Anchor points are selected according to the view, pose and visible partition of the target shot to generate a conditional source map and a spatial gating mask. Identity features and the state of the role at the end of the preceding shot are injected through an identity attention branch and a timing control branch. The first out-of-limit frame, abnormal partition and source label are determined according to the identity residual and boundary residual, only the hidden variables in the space-time repair map are updated, and the terminal state after acceptance is fed back to the subsequent shot. The application can improve the consistency of the role identity and the continuity across shots, and reduce repeated generation and consumption of computing resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence-generated content and computer vision technology, specifically to a method and system for generating short dramas based on multi-view character anchor points and camera chain feedback. Background Technology

[0002] With the development of deep learning, diffusion models, and multimodal generative models, technologies for automatically generating videos based on text, images, and other conditions are increasingly being applied to short dramas, animations, advertisements, and digital content production. In the short drama generation process, the script is typically parsed first to obtain descriptive text for multiple scenes or shots. Then, video clips corresponding to each shot are generated separately, and finally, the video clips are spliced ​​together according to the shot order. Since the same character often appears consecutively in multiple shots in a short drama, and different shots differ in shooting direction, character posture, expression, framing, degree of occlusion, and lighting conditions, maintaining the continuity of the same character's identity and movement across multiple shots is a crucial factor affecting the quality of automatically generated short dramas.

[0003] Chinese patent application CN120279470A discloses a method and related apparatus for obtaining frame sequences from text. This method obtains frame description text and layout information for each image frame based on input text, acquires image embeddings based on reference images of each character, and fuses the image embeddings, text embeddings, and layout information separately. Then, it generates a frame sequence including multiple image frames based on a spatiotemporal self-attention mechanism. This method limits the spatial position of characters within image frames through layout information and enhances the consistency of the same character across different image frames through image embedding of reference images, thus reducing feature interference between multiple characters to some extent. However, this patent primarily uses the overall image embedding of the character's reference image as a character condition. When the character in the target shot undergoes a significant change in perspective relative to the reference image, or when the character's face, hairstyle, clothing, and body area are obscured to varying degrees, the visible features contained in a single reference image may be insufficient to constrain the entire character area in the target shot. For example, a frontal reference image may not fully reflect the character's side profile or the texture of the clothing on the back, easily causing shifts in face shape, hairstyle, clothing patterns, or body proportions in different shots. At the same time, after encoding the reference image as a whole and fusing it with the layout information, it is usually impossible to clearly record the actual source of the conditions used by each character partition in the target shot. Therefore, when character anomalies occur in the generated result, it is difficult to determine whether the anomaly is caused by the mismatch of the reference image, the failure of the character spatial constraints, or the discontinuity of the state of adjacent shots.

[0004] Additionally, Chinese patent CN119583907A discloses a method, system, and device for scene-change-controllable text-based video generation. This solution utilizes a video generator based on a spatial layout fusion mechanism to generate video segments. Through a cross-segment scheduling mechanism, it inputs multiple video frames from the end of the previous video segment into a frequency-mixing control branch network. Using the high-frequency and low-frequency information of these multiple video frames, it generates subsequent video segments in an autoregressive manner, thereby improving the continuity between adjacent video segments. While this patent improves the scene and motion continuity of long videos by inheriting image information from the end of the previous video segment, its continuity control primarily focuses on the entire video frame or segment, without establishing traceable state dependencies for specific characters within a short drama shot. When there are shot transitions, changes in framing, alternating appearances of multiple characters, or significant posture changes of the same character in a short drama, directly inheriting the overall image information from the end of the previous video segment may continue to pass on character deviations, background interference, or occlusion information already present in the previous video segment to subsequent shots. In addition, the composition and scenes of different short drama shots may have significant differences, and the overall frame-level information mixing may also create unnecessary constraints on the areas that should be regenerated in the current shot.

[0005] In existing short drama generation processes, each shot is typically generated independently based on its own descriptive text. When inconsistencies in the identity characteristics of a character's area appear in the generated result, or when the position, posture, and movement direction of characters in adjacent shots are discontinuous, the entire shot usually needs to be regenerated. This approach struggles to distinguish the source of the anomaly and cannot only address the frame where the anomaly first appears and its corresponding character area. It can easily cause changes to background areas where no anomalies occurred, other character areas, and already satisfactory scene content, increasing not only the number of model inferences and computational resource consumption but also potentially introducing new visual deviations during the regeneration process.

[0006] Therefore, existing technologies still need to address the following technical issues: providing appropriate and clearly defined character constraints for different character regions in the target shot, taking into account the differences in the visibility of character features under different viewpoints, postures, and occlusion conditions; transmitting character-related postures and motion states between adjacent shots while maintaining the correspondence between the transmitted states and the specific preceding shots; distinguishing and locating character identity anomalies and shot boundary anomalies in the generated results; and performing targeted repairs on abnormal frames and abnormal character regions without altering normal screen areas, thereby improving the consistency of character identities and cross-shot motion states in short dramas and reducing the computational resource consumption caused by repeated generation of entire shots. Summary of the Invention

[0007] This invention aims to provide a short drama generation method and system based on multi-view character anchor points and camera chain feedback, in order to solve the technical problems in the existing short drama generation process, such as the easy drift of character identity characteristics under different viewpoints, postures and occlusion conditions, the discontinuous connection of character states in adjacent shots, the difficulty in identifying the source of anomalies, and the waste of computing resources caused by repeated generation of the entire shot.

[0008] To achieve the objectives of this invention, the following technical solution is adopted:

[0009] A short drama generation method based on multi-view character anchor points and camera chain feedback includes the following steps:

[0010] S1, parse the short drama script to obtain the shot nodes, and establish a shot chain with state-dependent edges according to the shared characters and shot connection relationship;

[0011] S2, partition the multi-view baseline map of the character, calibrate the viewpoints and standardize the coordinates to form a set of character anchor points containing partition features and visibility confidence;

[0012] S3, map the target character's pose to the standard coordinates, uniquely select the character anchor point for each target's visible partition, generate a character partition-condition source graph, and record the partition identifier, anchor point identifier, and spatial gating mask for each partition;

[0013] S4, generate target shots along the lens chain, inject the identity features of the corresponding role anchor point into the video generation network through the identity attention branch according to the spatial gating mask, and inject the posture-motion features obtained by the transformation of the end role state of the foreground shot into the video generation network through the temporal control branch. After the end role state of the foreground shot associated with the target shot through the state dependency edge is transformed into posture-motion features, it is injected into the temporal control branch.

[0014] S5, reverse map the character partition of the generated shot to the normal coordinates, compare it with the character anchor point recorded in the same partition to obtain the identity residual, compare the character state at the beginning of the target shot with the character state at the end of the foreground shot to obtain the boundary residual, and determine the first over-limit frame, abnormal partition and identity source, boundary source or composite source label.

[0015] S6. Based on the tags, a spatiotemporal restoration map is formed. The identity source calls the role anchor point recorded in the abnormal partition, the boundary source calls the end role state of the preceding lens, and the composite source calls both simultaneously. In each denoising stage, only the hidden variables in the spatiotemporal restoration map are updated. After acceptance, the terminal role state is transmitted to the subsequent lens and the accepted lens is stitched together.

[0016] As a further improvement, in step S1, the camera node records at least the character appearing, the target posture, the shooting angle, the character's movement direction, and the camera boundary; the state-dependent edge records the preceding camera identifier, the following camera identifier, the shared character identifier, the state item to be migrated, and the coordinate transformation parameters for converting the preceding camera coordinates to the following camera coordinates; a state version identifier is assigned to the end character state of the preceding camera, so that the subsequent camera generation and boundary residual calculation call the same end character state pointed to by the state version identifier.

[0017] As a further improvement, in steps S2 and S3, a character specification template including facial, hairstyle, clothing and body proportion partitions is established, each character partition is mapped to the character specification template, and the view vector, partition identity features, partition visibility mask and specification coordinate mapping relationship are recorded for each character anchor point; an anchor point score is generated based on view deviation, visibility confidence and partition coverage, and the same character anchor point is maintained in the same visible partition of the same target in consecutive frames until its anchor point score is lower than the retention threshold and the anchor point score of another candidate anchor point is higher than the switching threshold, wherein the switching threshold is higher than the retention threshold.

[0018] As a further improvement, in steps S3 and S4, a source map version identifier is generated each time a role partition-condition source map is established or updated, and a condition source snapshot at the time of target shot generation is saved. The condition source snapshot includes the source map version identifier and the partition identifier, anchor point identifier, spatial gating mask, and normalized coordinate mapping relationship corresponding to each target visible partition. The partition identity features pointed to by the anchor point identifier are mapped to the key features and value features of the identity attention branch. Before attention weight normalization and before attention score normalization, the attention score corresponding to the spatial position outside the target visible partition is set to an invalid value according to the spatial gating mask.

[0019] As a further improvement, in step S5, the generated shot is associated with the condition source snapshot saved when it is generated; for each generated partition, only its identity features are compared with the partition identity features pointed to by the anchor point identifier corresponding to the same partition identifier in the condition source snapshot, without reselecting other role anchor points; the target shot head role state and the front shot end role state pointed to by the state version identifier are time-aligned to obtain the position residual, attitude residual and motion residual respectively, and the boundary residual is determined based on the three.

[0020] As a further improvement, in steps S5 and S6, the frame that first exceeds the corresponding residual threshold is determined as the first over-limit frame according to the playback order of the shots; an identity source label is generated when only the identity residual exceeds the limit, a boundary source label is generated when only the boundary residual exceeds the limit, and a composite source label is generated when both the identity residual and the boundary residual exceed the limit; the source map version identifier, anchor point identifier, and status version identifier are written into the corresponding source label, and the consistency of the source map version identifier in the source label with the conditional source snapshot is checked before local regeneration.

[0021] As a further improvement, in step S6, the abnormal partition in the first out-of-limit frame is used as the initial repair area. The initial repair area is propagated along the time direction according to the optical flow of adjacent frames and the pose of the target character. After the consistency check is passed, the corresponding repair conditions are provided to the identity attention branch and the timing control branch according to the source label. The initial noise when the target shot is first generated and the original latent variables of each denoising stage saved during the initial generation of the target shot are reused. In each denoising stage, the repaired latent variables are written into the area within the spatiotemporal repair map. When the repaired identity residual and boundary residual do not exceed the limit, the acceptance is determined to be passed, and the subsequent shot is updated only when the state fingerprint of the terminal character state changes.

[0022] Another aspect of the present invention provides a short drama generation system based on multi-view character anchor points and camera chain feedback, comprising:

[0023] The camera chain construction module is used to parse the short drama script to obtain camera nodes and establish a camera chain with state dependency edges according to the shared characters and camera connection relationship; generate a state version identifier for the end character state of the accepted preceding camera, and associate the state version identifier with the corresponding state dependency edge;

[0024] The character anchor point construction module is used to partition, calibrate, and standardize coordinates of the character's multi-view baseline map to establish a set of character anchor points.

[0025] The Condition Source Binding module is used to map the target character's pose to the normalized coordinates, uniquely select the character anchor point for each target's visible partition, and generate a character partition-condition source graph that records the partition identifier, anchor point identifier, and spatial gating mask.

[0026] The dual-branch generation module is used to inject the identity features of the corresponding character anchor point into the identity attention branch according to the spatial gating mask, and to inject the character state at the end of the foreground shot into the temporal control branch after converting it into posture-motion features, so as to generate the target shot.

[0027] The residual attribution module is used to calculate the identity residual based on the character anchor points recorded in the same character partition, calculate the boundary residual based on the character state at the end of the forward camera pointed to by the state dependency edge, and determine the first over-limit frame, abnormal partition and source label.

[0028] The repair feedback module is used to form a spatiotemporal repair map based on the source tags. Based on the identity source tags, boundary source tags, or composite source tags, it calls the character anchor point, the end character state of the preceding shot, or a combination of the two as local regeneration conditions to implement local regeneration, and feeds back the accepted terminal character state to the subsequent shot.

[0029] As a further improvement, the conditional source binding module includes an anchor point scoring unit, an anchor point locking unit, and a source snapshot unit; the anchor point scoring unit is used to generate anchor point scores based on viewpoint deviation, visibility confidence, and partition coverage; the anchor point locking unit is used to control the retention or switching of anchor points within consecutive visible frames based on retention thresholds and switching thresholds; the source snapshot unit is used to save the source map version identifier, partition identifier, anchor point identifier, spatial gating mask, and canonical coordinate mapping relationship used when generating the target shot.

[0030] As a further improvement, the residual attribution module is used to associate the generated shot with the condition source snapshot saved at the time of its generation, using the character anchor point recorded in the same partition of the condition source snapshot as the identity residual comparison benchmark, and using the character state at the end of the preceding shot pointed to by the state version identifier as the boundary residual comparison benchmark; the repair feedback module includes a source consistency verification unit, a condition routing unit, a latent variable update unit, and a state propagation unit, which are respectively used to verify the source map version identifier, provide repair conditions according to the source label, update only the latent variables in the spatiotemporal repair map, and control the update of subsequent shots according to the state fingerprint changes of the terminal character state.

[0031] This invention establishes a shot chain with state-dependent edges for shot nodes in a short drama script, enabling the final character state of a preceding shot to serve as a temporal generation condition for subsequent shots, thus improving the continuity of character position, posture, and movement states between adjacent shots. By partitioning, calibrating, and standardizing coordinates of the character's multi-view reference image, it establishes a set of character anchor points containing partition features such as face, hairstyle, clothing, and body proportions, as well as visibility confidence. Furthermore, it selects suitable character anchor points for different character partitions based on the target character's viewpoint, posture, and visible area, thereby reducing the issues arising from using a single reference image for face shape and hairstyle. The system addresses issues such as clothing texture and body proportion drift. By using a character partitioning-conditional source map and spatial gating masks, the identity features of corresponding anchor points are injected into the target character area, reducing feature crosstalk between multiple characters and different character partitions. By calculating identity residuals and shot boundary residuals separately, the system can determine the first out-of-limit frame, abnormal partitions, and abnormal sources, and form corresponding spatiotemporal restoration maps based on identity sources, boundary sources, or combined sources. During the restoration process, only the latent variables within the spatiotemporal restoration map are updated, preserving normal character areas, background areas, and other accepted scene content, avoiding the introduction of new image deviations due to the regeneration of the entire shot. The terminal character state after restoration and acceptance continues to be passed to subsequent shots, thus forming a closed loop of "anchor point selection—dual-branch generation—residual detection—source determination—local restoration—state feedback." This improves the consistency of character identities and cross-shot continuity in the short drama while reducing the scope of repeated generation and the number of model inferences, thus reducing the computational resources and processing time required for video generation. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the short drama generation method based on multi-view character anchor points and camera chain feedback of the present invention.

[0033] Figure 2 This is a schematic diagram of the structure and main data flow of the short drama generation system of the present invention;

[0034] Figure 3 This is a schematic diagram illustrating the working principle of the role partitioning-condition source diagram, identity attention branch, and timing control branch of the present invention.

[0035] Figure 4 This is a schematic diagram of the process for attributing identity residuals and boundary residuals, spatiotemporal local repair, and lens chain state feedback in this invention.

[0036] Figure 5 This is a schematic diagram of the experimental verification process for the embodiments and comparative examples of the present invention;

[0037] Figure 6 This is a schematic diagram showing the average character identity consistency and average shot boundary residual distribution of embodiments of the present invention and various comparative examples. Detailed Implementation

[0038] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0039] I. Terminology Explanation

[0040] 1. A short drama script is a data file containing one or more of the following: plot text, character information, scene information, character dialogue, character actions, and shot descriptions. A short drama script can be natural language text, a structured script file, or a hybrid file composed of natural language text and structured fields.

[0041] 2. A shot node refers to a data processing unit that corresponds to a video segment to be generated, defined according to the script of the short drama. A shot node must at least have a shot identifier and can further record the shot's start and end times, duration, target frame rate, aspect ratio, shot type, shooting direction, camera movement, scene description, character identifier, character posture, character action, character expression, occlusion relationship, and connection relationship with other shots.

[0042] 3. A shot chain is a directed data structure consisting of shot nodes and state-dependent edges. A shot chain can be a linear chain or a directed acyclic graph with branches. When a short drama has flashbacks, interludes, parallel narratives, or multiple subsequent shots corresponding to the same preceding shot, the shot chain can include multiple state-dependent branches.

[0043] 4. A state-dependent edge is a data structure that records the character states that need to be transferred between a preceding shot and a subsequent shot, as well as the state transition relationships. A state-dependent edge records at least the preceding shot identifier, the subsequent shot identifier, the shared character identifier, and the state item to be transferred. It can also record coordinate transformation parameters, state version identifier, generation time, state validity range, and state verification value.

[0044] 5. Multi-view character anchor points refer to character features extracted from one or more baseline images of the same character, corresponding to the character's viewpoint and character sections. Multi-view character anchor points are not limited to a single complete character image, but can correspond to sections such as the character's face, hairstyle, clothing, or body proportions. Different sections can originate from baseline images from different viewpoints.

[0045] 6. Standardized coordinates refer to a standardized coordinate system used to uniformly represent the position of different character sections in different reference images and shots. Standardized coordinates can be two-dimensional normalized coordinates, local coordinates constructed based on human keypoints, human surface parameter coordinates, or three-dimensional character template coordinates. The purpose of using standardized coordinates is to ensure that character sections extracted from different viewpoints can correspond to the same character structural position.

[0046] 7. The Character Partition-Condition Source Graph is a data structure that records the correspondence between each target character partition in a target shot and its generation condition source. It can be implemented using a graph structure, index table, key-value table, or data object with edge information. Each target character partition is uniquely associated with a character anchor point, and the partition identifier, character anchor point identifier, and spatial gating mask of the target character partition are recorded.

[0047] 8. The identity attention branch refers to the conditional processing branch used in the video generation process to inject the identity features of the character anchor point into the target character partition according to the spatial gating mask. The identity features may include not only facial recognition features, but also hairstyle outline, clothing texture, body proportions and other features used to distinguish the appearance of the character.

[0048] 9. The timing control branch refers to the conditional processing branch that converts the role state of the front-end lens into a pose-motion feature that can be used by the target lens, and injects the pose-motion feature into the video generation network.

[0049] 10. Identity residual refers to the difference obtained by comparing the target character partition in the generated shot, after being inversely mapped to normalized coordinates, with the actual character anchor point used when generating the target character partition. Identity residual can consist of one or more of the following: feature distance, structural difference, texture difference, and scale difference.

[0050] 11. Boundary residual refers to the difference between the character state at the beginning of the target shot and the character state at the end of the preceding shot, as indicated by the state-dependent edge, after coordinate alignment. The beginning of the target shot can be the first frame of the target shot or multiple consecutive frames in the beginning part of the target shot.

[0051] 12. A spatiotemporal inpainting graph is a data structure that simultaneously defines the interval of frames to be inpainted and the spatial regions of each frame to be inpainted. A spatiotemporal inpainting graph can be represented as a binary mask arranged by frames, a gray-scale weighted mask, or a sparse region index.

[0052] 13. Terminal character state refers to the character's position, posture, motion, and occlusion status extracted from one or more video frames at the end of an accepted shot. Terminal character state is used to control the generation of subsequent shots, rather than simply copying the complete last frame of the preceding shot directly to the subsequent shot.

[0053] 14. A condition source snapshot refers to an immutable data object formed by permanently recording the character partition-condition source map, character anchor point version, and preceding shot state version when the target shot begins to be generated. Through the condition source snapshot, condition data from the same source can be used during the generation, detection, and repair phases.

[0054] II. System Structure

[0055] like Figure 2 As shown, the short drama generation system includes a shot chain construction module, a character anchor point construction module, a condition source binding module, a dual-branch generation module, a residual attribution module, a repair feedback module, a shot acceptance and splicing module, and a data storage module.

[0056] The data storage module includes a short drama script library, a character anchor point library, a shot status library, and a generation task library. The short drama script library stores the original short drama scripts and the parsed shot nodes; the character anchor point library stores the character multi-view baseline map, character partitions, partition features, view vectors, visibility confidence, and character anchor point versions; the shot status library stores the terminal character status and status version identifier for each accepted shot; and the generation task library stores the character partition-condition source map, condition source snapshot, generation parameters, initial noise, latent variables at each denoising stage, residual detection results, and repair tasks.

[0057] The shot chain construction module is used to parse the short drama script, divide the script into shot nodes, and establish state dependency edges based on whether adjacent shots share characters, whether character actions are continuous, and whether there is a sequential relationship between shots. The character anchor point construction module is used to partition the character multi-view baseline map, calibrate the viewpoints, and standardize the coordinates to form a character anchor point set.

[0058] The condition source binding module maps the character pose of the target shot to standardized coordinates, selects character anchor points based on the viewpoint, visibility, and spatial coverage of the target character partition, and generates a character partition-condition source graph. The dual-branch generation module includes an identity attention branch, a temporal control branch, and a video generation network, which are used to generate the target shot after processing the character identity conditions and shot temporal conditions respectively.

[0059] The residual attribution module includes an identity residual detection unit, a boundary residual detection unit, and an anomaly source determination unit. The repair feedback module is used to generate a spatiotemporal repair graph, perform condition source consistency verification, call the generation conditions corresponding to the anomaly source, and update the latent variables in the anomaly region through the local regeneration unit.

[0060] The shot acceptance and stitching module is used to determine whether the repaired identity residuals and boundary residuals meet the acceptance criteria. For shots that pass acceptance, it extracts the terminal role status and stitches the accepted shots together according to the shot chain sequence. These modules can be deployed in the same electronic device or distributed across a script processing server, image processing server, video generation server, and task scheduling server. The electronic device may include a processor, graphics processor, memory, and network interface. The memory stores a computer program that causes the processor to execute the method of this invention.

[0061] III. Overall Method Flow

[0062] like Figure 1 As shown, the method of the present invention includes steps S1 to S6. The system first establishes a shot chain and a set of character anchor points, and then determines a unique source of conditions for each character partition in the target shot. During the video generation stage, the system injects character identity conditions and the state of preceding shots using the identity attention branch and the temporal control branch, respectively. After generation, the system calculates identity residuals and boundary residuals using snapshots of fixed source conditions from the generation process, determines the source of anomalies based on the residuals, performs local regeneration only for abnormal time intervals and abnormal spatial regions, and transmits the accepted terminal character state to subsequent shots.

[0063] 1. Step S1: Analyze the short drama script and establish the camera chain.

[0064] The shot chain construction module reads short drama scripts from the short drama script library and identifies scene titles, character names, character dialogues, action descriptions, shot descriptions, and time sequence information in the scripts. For natural language scripts that have not been pre-divided into shots, shots can be divided based on scene changes, character entrances and exits, action phase changes, dialogue rounds, and preset shot durations; for scripts that already contain shot information, each shot can be directly converted into a shot node.

[0065] Each shot node is assigned at least one unique shot identifier. Shot nodes can also record the set of character identifiers, the predicted area of ​​the character in the frame, the shot size, the shooting direction, the camera azimuth angle, the camera tilt angle, the character's orientation, the character's posture, the character's expression, the stage of the action, occlusion relationships, scene lighting, and video generation prompt text.

[0066] The camera chain construction module determines whether a state dependency exists between two camera nodes. When a subsequent camera continues to depict the same character's actions, position changes, turning, walking, picking up objects, or facial expressions from the previous camera, a state dependency edge is established between the two camera nodes. This state dependency edge records at least the preceding camera node, the target camera node, the shared character identifier, and the state items to be transmitted. For example, state items may include the character's screen position, keyframes, body orientation, direction of movement, speed of movement, visible area, and occlusion status.

[0067] When adjacent shots have different image resolutions, framing, or camera coordinates, the state-dependent edge also records coordinate transformation relationships. These coordinate transformation relationships can be determined based on the image ratio, character bounding box, human center point, human key points, and shooting direction, and are used to transform the character's state in the preceding shot to the coordinate system used by the target shot.

[0068] After the preceding camera passes acceptance testing, the system generates a state version identifier based on the preceding camera identifier, shared role identifier, terminal frame range, and terminal role state. The state version identifier is associated with state dependency edges, ensuring that subsequent camera generation and boundary residual detection use the same version of the preceding camera state. If the preceding camera has not yet passed acceptance testing, its subsequent cameras remain in a waiting state, or only perform preprocessing operations that do not depend on the preceding camera state.

[0069] 2. Step S2: Establish a multi-view character anchor point set

[0070] The character anchor point construction module obtains multi-view reference images for each character. These reference images can include front view, left side view, right side view, back view, top view, bottom view, and reference images with different expressions or action states. It is not required that each character have all viewpoints; this invention can be implemented as long as character features can be extracted from at least one reference image. When the number of reference images is large, the coverage of character features across different viewpoints can be further improved.

[0071] The character anchor point construction module performs character detection and semantic segmentation on each baseline image, dividing the character into facial, hairstyle, clothing, and body proportion partitions. The facial partition can further include the eyes, nose, mouth, and facial contours; the clothing partition can further include the upper garment, lower garment, accessories, and shoes; the body proportion partition can be represented by the relative positions between key points of the head, shoulders, torso, arms, and legs. The above further divisions are optional implementation methods.

[0072] The character anchor point construction module calibrates the viewpoint of the reference image based on human body key points, facial key points, and shooting direction, obtaining viewpoint vectors representing horizontal rotation angle, pitch angle, and roll angle. For back views or reference images with completely obscured faces, calibration can be completed based on head and shoulder contours, body orientation, or viewpoint labels pre-recorded during shooting.

[0073] Subsequently, standardized coordinates are established using a standard human body template or a standard baseline image of the character itself. The character anchor point construction module aligns the character sections in each baseline image to the standardized coordinates based on the correspondence between human body key points, the correspondence between partition boundaries, or the mapping relationship of the human body surface. Through this process, the upper garment section in the front view and the upper garment section in the side view can be identified as the same type of section of the same character, while retaining their respective viewpoint features and visible areas.

[0074] For each character anchor point, the character anchor point library records at least the character identifier, anchor point identifier, partition identifier, source reference map identifier, view vector, canonical coordinate transformation information, partition features, and visibility confidence. Partition features can be obtained from the corresponding image feature encoder. Facial partitions can use identity features that are discriminative of the character; hairstyle partitions can use contour and texture features; clothing partitions can use color, pattern, and local texture features; body proportion partitions can use normalized distance between keypoints and contour features.

[0075] Visibility confidence indicates whether a given baseline image adequately displays the corresponding character's area. For example, facial visibility confidence is higher in a frontal baseline image and lower in a back baseline image; body proportion area confidence is higher in a full-body baseline image and lower in a baseline image that only includes the head. Visibility confidence can be determined by occlusion percentage, image sharpness, area of ​​the area, and keypoint completeness.

[0076] After anchor point construction is completed, an anchor point set version identifier is generated for the same batch of character anchor points. When adding, deleting, or replacing the baseline map, or when re-partitioning and standardizing coordinate alignment, a new anchor point set version is generated to prevent already generated shots from being replaced by new character anchor points during residual detection.

[0077] 3. Step S3: Establish Role Partitions - Condition Source Diagram

[0078] like Figure 3 As shown, the condition source binding module obtains the shooting direction, character pose, framing, and occlusion information of the target shot node, and maps the predicted pose of the target character to the standardized coordinates established in step S2. After mapping, the system determines the visible partitions of each target in the target shot and their expected visible range.

[0079] For each target visible partition, the conditional source binding module reads candidate character anchors from the character anchor library whose character identifier and partition identifier are the same. For example, if the character in the target shot is standing sideways, and the target visible partition includes the side face, hairstyle, clothing, and body proportions, then the face partition is preferentially selected from face anchors that are close to the target's side view. The clothing partition can be selected from another reference map that can fully display the side texture of the clothing, and the body proportion partition can be selected from a reference map that clearly displays the full body structure. Thus, different partitions of the same target character can use character anchors from different reference maps, but each target visible partition has only one determined character anchor source in a single shot generation task.

[0080] In one possible implementation, the system calculates the fit score of candidate character anchor points based on viewpoint deviation, visibility confidence, spatial coverage, and image quality.

[0081] ;

[0082] In the formula, Indicates role target partition With the The compatibility score between the anchor points of each candidate character; This indicates the viewpoint fit degree obtained based on the deviation between the target camera viewpoint and the candidate character's anchor point viewpoint; Indicates the visibility confidence level of candidate role anchors; This indicates the degree to which the candidate character's anchor point covers the target's visible region in the normalized coordinates; Image quality evaluation value representing the anchor point of the candidate character; to These represent the weights of each evaluation item, and the sum of all weights is 1.

[0083] Viewpoint fit can be obtained by normalizing the angle between the target viewpoint vector and the character anchor point viewpoint vector. Spatial coverage can be obtained by the degree of overlap between the effective visible area of ​​the character anchor point and the visible area of ​​the target in normal coordinates. Image quality evaluation values ​​can be determined based on sharpness, noise, exposure, and occlusion.

[0084] In one example, the weights for viewpoint fit, visibility confidence, spatial coverage, and image quality are set to 0.35, 0.30, 0.25, and 0.10, respectively. These values ​​are merely one set of implementation parameters and can be adjusted based on character type, visual style, and training data.

[0085] To prevent frequent switching of character anchor points due to slight fluctuations in adaptation scores within consecutive shots, the condition source binding module employs a hysteresis mechanism consisting of a retention threshold and a switching threshold, where the switching threshold is higher than the retention threshold. If a target partition already has a selected character anchor point in the previous shot or generation round, and the current adaptation score of that character anchor point is not lower than the retention threshold, then that character anchor point continues to be used. Switching to the other candidate character anchor point only occurs when its current adaptation score is lower than the retention threshold, and the adaptation score of another candidate character anchor point is higher than the switching threshold.

[0086] For example, the retention threshold can be set to 0.62, and the switching threshold to 0.74. Using two different thresholds can prevent characters from exhibiting abrupt changes in clothing texture or facial details due to repeated anchor point switching from similar viewpoints. If no candidate character anchor point reaches the switching threshold, the original character anchor point can be retained with its injection weight reduced, or the corresponding target partition can be marked as a partition to be supplemented with anchor points.

[0087] After selecting the character anchor point, the condition source binding module generates a spatial gated mask based on the predicted position of the target character partition in the target shot. The spatial gated mask can be generated from the character bounding box, human keypoints, partition outline, and estimated occlusion area, and then converted to the spatial resolution corresponding to each feature layer of the video generation network. The spatial gated mask can be a binary mask, or gradient weights can be set at the partition edges to reduce discontinuities at the boundaries of the generated region.

[0088] The conditional source binding module forms a character partition-conditional source graph consisting of target character partitions, character anchor points, and spatial gating masks. Each target character partition node records the target shot identifier, character identifier, partition identifier, and target spatial region; each source node records the character anchor point identifier, anchor point set version, and source reference map identifier; the source edge connecting the two records the standard coordinate transformation information and the spatial gating mask identifier.

[0089] The character partitioning-conditional source map is used not only to generate target shots but also for subsequent identity residual detection and local repair. Before the generation task begins, the system generates a source map version identifier based on the target shot identifier, anchor set version, anchor point identifiers of each character partition, spatial gating mask, and preceding shot status version, and forms a conditional source snapshot.

[0090] The condition source snapshot records at least the following: source image version identifier, target lens identifier, character identifier, partition identifier, character anchor point identifier, anchor point set version, spatial gating mask, standard coordinate transformation information, leading lens identifier, and leading lens status version. This snapshot remains unchanged throughout a single generation, detection, and repair task. If the user changes the character baseline image during generation, the system will not directly replace the current task's anchor point with the new one. Instead, it will create a new character partition—condition source image—after completing or terminating the current task.

[0091] Through the above processing, the system can determine which character segment in the target shot was generated by which character's anchor point, thus providing a traceable source of conditions for the generation result detection and anomaly repair. This processing differs from simply mixing features from multiple reference images and inputting the whole into the video generation model, and can reduce conditional interference between different viewpoints, different segments, and different characters.

[0092] 4. Step S4: Generate the target shot using the identity attention branch and the temporal control branch.

[0093] The dual-branch generation module reads snapshots from the condition source and establishes an identity attention branch and a timing control branch for the target shot. The two branches handle character identity information and cross-shot timing state respectively, and are injected into different condition interfaces of the video generation network to avoid mixing identity appearance and motion state into an indistinguishable single condition.

[0094] First, the identity attention branch reads the character anchor features bound to each target character partition. The character anchor features output by the image feature encoder can be converted into key and value features used by the attention mechanism via a linear mapping layer. The query features generated by the target shot latent variables only undergo attention operations with the key and value features within the region allowed by the corresponding spatial gating mask.

[0095] Spatial gating masks are applied before attention weight normalization. Specifically, for query locations that do not belong to the target character's partition, the attention score between the query location and the corresponding character's anchor point is set to an invalid or extremely low value, and then the remaining attention scores are normalized. Thus, facial anchor points mainly affect the target facial area, and clothing anchor points mainly affect the target clothing area, without injecting the clothing texture of character A into character B or the background area.

[0096] When multiple characters are present in the same frame, the system generates spatial gating masks for each character and each area. If two masks overlap, their priority is determined based on the occlusion relationship, the visibility confidence of the area, and the character depth order. The mask weight of the occluded character in the overlapping area is reduced or set to zero, while the occluded character retains a higher weight in the overlapping area. For character outline boundaries, a narrow transition area can be set to allow for a smooth change in weight between adjacent areas.

[0097] Identity attention branches can be placed in multiple spatial feature layers of the video generation network. Lower-resolution feature layers inject body proportions and overall contour features, while higher-resolution feature layers inject facial, hairstyle, and clothing texture features. This hierarchical injection approach helps to simultaneously preserve both the overall structure of the character and local identity details.

[0098] The timing control branch reads the state version of the preceding camera pointed to by the state dependency edge from the camera state library. The terminal character state of the preceding camera can be extracted from a preset number of video frames at the end of the preceding camera, such as the character bounding box center, human key points, body orientation, limb movement direction, movement speed, and occlusion status from the last 4, 8, or 16 frames.

[0099] The system transforms the terminal character state of the foreground camera to the target camera coordinates based on the coordinate transformation relationship recorded by the state-dependent edges. When the foreground and target cameras have different framing, the human body keypoints can be normalized to the character bounding box coordinates first, and then scaled and translated according to the expected bounding box of the character in the target camera. When the shooting directions of the foreground and background cameras are different, the body orientation and motion vector can be rotated and transformed according to the camera orientation difference.

[0100] The transformed character state is encoded as pose-motion features. Pose features can be represented using human keypoint heatmaps, skeleton connection maps, body orientation maps, or partitioned position maps; motion features can be represented using motion direction fields, velocity feature sequences, or trajectories estimated from multiple frames at the end. A temporal control branch injects these pose-motion features into the temporal feature layer of the video generation network or an additional control network to constrain the character's position, pose, and motion trend at the beginning of the target shot.

[0101] The identity attention branch primarily answers which character and what identity details should be presented in the target area, while the temporal control branch primarily answers from what posture and motion state the character should begin with. Both branches work together in the video generation network, but their input sources, spatial scope, and technical roles remain distinct.

[0102] During diffusion-based video generation, the generation task library stores the random seed used for the current target shot, the initial noise, and the latent variables for each denoising stage. The video generation network starts with the initial noise and gradually obtains the target shot according to the preset denoising stages. In each denoising stage that requires identity condition injection, the identity attention branch reads the anchor features and spatial gating mask from the condition source snapshot; in each denoising stage that requires temporal condition injection, the temporal control branch reads the predecessor state version specified by the condition source snapshot.

[0103] When a character anchor point, spatial gating mask, or precursor state version is externally updated during generation, the current generation task still uses the version recorded in the snapshot. This ensures that the generated footage maintains a definite correspondence with the actual source of the conditions it uses, preventing the misuse of updated character anchor points or precursor states during subsequent detection phases.

[0104] For the first shot in a shot chain, if there is no preceding shot, the initial character state can be constructed from the initial posture and initial motion description in the script, and an initial state version identifier can be assigned. For scene transition shots that do not require continuous action, state dependency edges can be marked as weak dependencies, only transmitting the character's orientation, body proportions, or visible partitions, without transmitting specific screen positions and movement speeds.

[0105] After the video generation network completes all denoising stages, it outputs a target shot consisting of multiple video frames and stores the target shot in association with the snapshot of the condition source at the time of generation. This association remains unchanged during subsequent identity residual detection, boundary residual detection, and local repair processes.

[0106] 5. Step S5: Detect identity residuals and boundary residuals and determine the source of anomalies.

[0107] like Figure 4As shown, the residual attribution module processes the generated shots through an identity residual detection unit and a boundary residual detection unit, respectively. The two detection units evaluate the consistency of character identity and the continuity of state across shots, respectively, avoiding the inability to identify the source of anomalies due to the use of a single comprehensive score.

[0108] The identity residual detection unit first performs character detection, character tracking, and character segmentation on each video frame of the generated shot to obtain the generated character segmentation in each frame. Based on the character key points, segmentation contours, and target character pose, the system back-maps the generated character segmentation to the standardized coordinates established in step S2.

[0109] During the inverse mapping process, facial regions can be aligned based on key points such as the corners of the eyes, the tip of the nose, the corners of the mouth, and facial contours; clothing regions can be aligned based on key points such as the shoulders, hips, elbows, and knees; hairstyle regions can be aligned based on the head contour and facial direction; and body proportion regions can be normalized based on the length of each skeletal segment. After alignment, the impact of differences in viewpoint and pose on feature comparison is reduced.

[0110] The system reads the character anchor point actually used when generating the character partition based on the snapshot of the conditional source associated with the generated shot, instead of reselecting the character anchor point with the highest adaptation score during the detection phase. For example, if the facial partition of the target's side shot uses anchor point A2 during generation, even if anchor point A3 is more similar to the generated result during detection, the generated result should still be compared with anchor point A2. Therefore, the identity residual can accurately reflect the degree of deviation of the generated result from the generation conditional source, avoiding the masking of identity drift by reselecting a more similar reference image.

[0111] Identity residuals can be calculated separately for different role zones. Facial identity residuals can be determined based on the distance between facial identity features; hairstyle identity residuals can be determined based on differences in outline, color, and texture; clothing identity residuals can be determined based on differences in local texture, pattern position, and color distribution; and body proportion identity residuals can be determined based on differences in head-to-body ratio, shoulder width, torso length, and limb length.

[0112] In one implementation, the partition identity residual is represented as:

[0113] ;

[0114] In the formula, Indicates role partitions In the Frame identity residual; This indicates the difference in feature encoding results; Indicates differences in the outline or structure of a partition; Indicates differences in color or texture; , and These represent the residual weights used for the corresponding partitions, and the sum of the three is 1.

[0115] Different partitions can be assigned different weights. For example, the face partition can increase the weight of feature encoding differences, the clothing partition can increase the weight of texture differences, and the body proportion partition can increase the weight of structural differences. Each partition can also have different identity residual thresholds.

[0116] The boundary residual detection unit extracts the character state from the beginning of the target shot and reads the preceding shot state version from the condition source snapshot record. Based on the coordinate transformation relationship in the state dependency edges, the system transforms the character state at the end of the preceding shot and the character state at the beginning of the target shot to the same coordinate reference before comparison.

[0117] Boundary residuals can include at least position residuals, pose residuals, and motion residuals. Position residuals represent differences in the position of the character's center, key points on the human body, or specified body sections; pose residuals represent differences in skeletal angles, body orientation, or pose characteristics; and motion residuals represent differences in motion direction, speed, or motion trend. For short drama shots that involve cuts, the absolute screen coordinates of the characters are not required to be the same; instead, the transformed states are compared based on the coordinate transformation relationships recorded by the state-dependent edges.

[0118] The system detects identity residuals and boundary residuals sequentially according to video frames, identifying the video frame that first exceeds the corresponding threshold. For boundary residuals, the detection range can be focused on the beginning of the target shot; for identity residuals, all video frames of the target shot can be detected. When a certain partition exceeds the threshold in multiple consecutive frames, the earliest exceeding frame is taken as the first exceeding frame.

[0119] The anomaly source determination unit determines the identity source, boundary source, or composite source label based on the residual type. If the identity residual of an anomaly partition exceeds the identity threshold, while the boundary residual of the corresponding time period does not exceed the boundary threshold, it is determined to be an identity source; if the boundary residual exceeds the boundary threshold, while the identity residual of each role partition does not exceed the identity threshold, it is determined to be a boundary source; if both the identity residual and the boundary residual exceed the corresponding threshold, or if both appear consecutively within a preset associated time window, it is determined to be a composite source.

[0120] For each anomaly, the system records the target camera identifier, the first frame exceeding the limit, the anomaly character identifier, the anomaly partition identifier, the anomaly region, the residual type, the residual value, and the anomaly source label. The anomaly source label also records the source image version, the character anchor point identifier, and the predecessor state version.

[0121] For example, if the pattern on the character's clothing changes starting from frame 28, but the position and posture of the target shot's head are continuous with the foreground shot, it is determined to be the source of identity; if the character suddenly changes from walking to ...

[0122] By comparing the generated results with the actual source used during generation and retaining the source version in the anomaly label, the residual attribution module can pinpoint the anomaly to a specific time, specific role partition, and specific condition source, providing a definite data basis for subsequent local repair.

[0123] 6. Step S6: Generate a spatiotemporal repair map, locally regenerate, and feed back the terminal role status.

[0124] The repair feedback module generates a spatiotemporal repair map based on the initial out-of-limit frame, anomalous partitions, and anomalous source tags. First, the anomalous partition mask from the initial out-of-limit frame is used as the initial repair area. For the edge transition region containing the anomalous partition, moderate dilation and smoothing can be performed on the initial repair area, but the dilated area should still be limited to the allowed area of ​​the corresponding character to avoid covering other characters or irrelevant backgrounds.

[0125] Subsequently, the system utilizes optical flow, character tracking results, and partition contour changes between adjacent video frames to propagate the initial repair area to subsequent frames. If anomalies may have gradually accumulated from earlier frames, it can also propagate backwards to a preset number of frames before the first out-of-limit frame. During propagation, the repair area for each frame is updated based on the pixel displacement estimated by optical flow, and the propagation results are constrained using character partition contours, forming a spatiotemporal repair map that simultaneously includes frame position and spatial position.

[0126] When an abnormal character's partition is briefly occluded, the system can propagate the repair area across the occluded frames based on the partition's trajectory before and after occlusion. However, in fully occluded areas, the repair weight is reduced or set to zero. Therefore, the repair area can follow the character's movement, rather than being fixed at the same screen coordinates across all frames.

[0127] Before performing a partial regeneration, the repair feedback module performs a condition source consistency check. The system reads the source image version, character anchor point identifier, and predecessor state version from the anomaly source tag and compares them with the condition source snapshot associated with the generated shot. Only when the corresponding identifiers match will the condition be invoked to perform a partial regeneration.

[0128] If the current character anchor library has been updated, but the condition source snapshot contains the original character anchors and their characteristics, then the original character anchors saved in the snapshot will continue to be used. If the original character anchor data is no longer readable, the system will stop the current repair task and rebuild the condition source graph, instead of directly calling new character anchors without recording them. The same handling applies when the predecessor state version is inconsistent. This consistency check prevents the generation, detection, and repair phases from using data from different sources.

[0129] For the identity source label, the local regeneration unit calls the role anchor point recorded in the condition source snapshot of the abnormal partition, and the identity attention branch re-injects the anchor point features into the region defined by the spatiotemporal repair map. At this time, the injection intensity of the corresponding role anchor point in the abnormal region can be increased without changing the original attitude-motion conditions of the temporal control branch.

[0130] For boundary source labels, the local regeneration unit calls the character state at the end of the preceding shot recorded in the condition source snapshot, and the temporal control branch regenerates the pose-motion features and injects them into the frame interval corresponding to the spatiotemporal restoration map. At this time, the original conditions of the identity attention branch are maintained so that the character identity features do not undergo unnecessary changes due to boundary state restoration.

[0131] For composite source tags, the local regeneration unit simultaneously calls the role anchor point of the abnormal partition and the role state at the end of the foreground camera, so that the identity attention branch and the timing control branch work together on the repair area.

[0132] The generation task library stores the random seed, initial noise, and latent variables used in the original generation task, as well as the latent variables for each denoising stage. During local regeneration, the system reuses the original initial noise and original latent variables instead of regenerating all video content randomly. In each denoising stage, the spatiotemporal restoration map defines the allowed locations of latent variables to be updated, and latent variables outside the spatiotemporal restoration map are locked.

[0133] In one implementation, the locally updated latent variables are represented as follows:

[0134]

[0135] In the formula, Indicates the first The latent variables after local updates in the denoising stage; This represents a spatiotemporal restoration map adapted to the current feature resolution. This represents a latent variable that is recalculated under the corresponding anomaly source conditions; Indicates that the original generated task was at the 1st rank. The hidden variables saved in each denoising stage; This indicates multiplication at corresponding positions.

[0136] when When using a binary mask, the recalculated latent variables are used within the repair area, while the original latent variables are used outside the repair area; when When using a grayscale weighted mask, the edges of the repaired area can smoothly transition between the original latent variables and the recalculated latent variables. Regardless of the mask used, the system does not recalculate or replace the final latent variables outside the spatiotemporal repair map, thus preserving the normal background, other characters, and the already satisfactory image content.

[0137] After local regeneration is completed, the residual attribution module recalculates the identity residual and boundary residual for the repaired area and adjacent boundary areas. The target shot passes acceptance only if the identity residual does not exceed the corresponding identity threshold and the boundary residual does not exceed the corresponding boundary threshold. If residuals still exceed limits, the system updates the spatiotemporal repair map based on the latest first-time exceedance frame and abnormal partition, and performs local regeneration again within a limited number of attempts. If acceptance is still not passed after reaching the maximum number of repair attempts, the target shot can be marked for manual review or the repaired area can be expanded, but this does not affect the implementation of the local repair mechanism of this invention.

[0138] After the target shot passes acceptance, the shot acceptance and stitching module extracts the terminal character state from one or more video frames at the end of the target shot. The terminal character state includes at least one of the following: character position in the frame, character bounding box, human key points, body orientation, direction of movement, speed of movement, visible area, and occlusion status.

[0139] The system standardizes and quantifies each state item in the terminal character's state and generates a terminal state fingerprint. The terminal state fingerprint can generate a hash value based on the camera identifier, character identifier, terminal frame interval, standardized character position, pose key points, movement direction, movement speed, and occlusion status. This fingerprint is used to determine whether the local repair has actually changed the terminal character state that needs to be passed to subsequent shots.

[0140] If the repaired terminal state fingerprint is the same as before the repair, it indicates that the repair did not change the terminal state passed from the camera chain to subsequent cameras, and there is no need to regenerate the completed subsequent cameras. If the terminal state fingerprint changes, a new state version identifier is generated and notified to subsequent cameras along the state dependency edges. Subsequent cameras that have not yet been generated directly adopt the new state version; subsequent cameras that have been generated but not yet accepted update their boundary detection tasks; and subsequent cameras that have been accepted first recalculate the boundary residuals, and only initiate local repair when the boundary residuals exceed the limits.

[0141] Therefore, the present invention does not regenerate all subsequent lenses when any local change occurs in the front lens, but uses whether the terminal state fingerprint changes as the state propagation condition to reduce the regeneration of invalid subsequent lenses.

[0142] After all target shots have passed acceptance, the shot acceptance and stitching module stitches the accepted shots according to the playback order specified in the shot chain. During stitching, audio synchronization, transition insertion, and encoding format unification can be further performed, but the above processing does not change the core technical approach of this invention, which maintains character consistency through multi-view character anchor points and shot chain feedback.

[0143] IV. Specific Application Examples

[0144] (I) Experimental Objective

[0145] This experiment is used to verify the technical effectiveness of the short drama generation method based on multi-view character anchor points and camera chain feedback in the following aspects:

[0146] 1. Verify whether uniquely binding multi-view character anchor points by character partition can improve the consistency of character identity under different viewpoints and different occlusion conditions;

[0147] 2. Verify whether passing the end-character state of the foreground camera to the target camera through state-dependent edges can reduce abrupt changes in position, attitude, and motion state between adjacent cameras;

[0148] 3. Verify whether the source of anomalies can be accurately determined based on identity residuals and boundary residuals, and whether the character anchor point, the character state at the end of the foreground shot, or a combination of the two can be accurately selected as the repair condition;

[0149] 4. Verify whether updating only latent variables within the abnormal region using the spatiotemporal restoration map can reduce the amount of restoration processing and the inference time of the video generation model;

[0150] 5. Verify whether the condition source snapshot, character anchor point lag switching, and terminal status fingerprint can reduce condition source mismatch, frequent character anchor point switching, and invalid subsequent shot regeneration, respectively.

[0151] (II) Experimental System and Operating Conditions

[0152] like Figure 2 As shown, the experimental system adopts a short drama generation system, which includes a shot chain construction module, a character anchor point construction module, a condition source binding module, a dual-branch generation module, a residual attribution module, a repair feedback module, a shot acceptance and splicing module, and a data storage module.

[0153] The experimental system was deployed on a computing device with a multi-core processor, 64 GB of RAM, and a 24 GB VRAM graphics processor. The operating system, video generation network, character segmentation model, pose recognition model, character feature encoding model, and optical flow estimation model remained consistent across all experimental groups. All experimental groups used the same model weights, video resolution, video frame rate, number of denoising stages, prompt text, and random seed; only the technical features corresponding to the comparison model were changed.

[0154] The video generation parameters are as follows: video resolution is 576×1024 pixels, using a portrait orientation; video frame rate is 24 frames / second; the duration of a single shot is 4 seconds, and each shot includes 96 frames; the number of denoising stages in the video generation network is 30; the maximum number of times each local repair is allowed to be performed is 2; the character anchor point retention threshold is 0.62, and the switching threshold is 0.74; the terminal character state of the foreground shot is extracted from the last 8 frames; the head character state of the target shot is extracted from the first 8 frames; each experimental group uses 3 different random seeds for repeated runs.

[0155] (III) Experimental Dataset

[0156] The experimental dataset includes 20 short drama scripts. According to... Figure 1 Step S1 shows the parsing of the short drama script, which establishes 60 three-shot shot chains. Each shot chain includes foreground shots, perspective switching shots, and action transition shots, for a total of 180 shots.

[0157] Each camera chain has at least one shared character, totaling 12 characters. Each character is equipped with 5 multi-view reference images: a front view, a left side view, a right side view, a back view, and a three-quarter side view. Some reference images are equipped with hand occlusion, hair occlusion, partial clothing occlusion, or prop occlusion to verify the effect of character anchor point selection under different visibility conditions.

[0158] Each shot was generated using 3 sets of random seeds. Therefore, each experimental group obtained 540 shot results, corresponding to 180 complete three-shot generation results. The total number of video frames processed in each experimental group was 51,840 frames.

[0159] To reduce evaluation bias, the character partitions in the experimental dataset were initially labeled by the character segmentation model, and then two labelers separately checked the facial, hairstyle, clothing, and body proportion partitions. If the two labelers' conclusions differed, a third labeler reviewed the data. Evaluators did not see the experimental group names during the evaluation process.

[0160] (iv) Evaluation indicators and statistical methods

[0161] 1. Consistency of role identity

[0162] Role identity consistency is used to evaluate the similarity of identity features between generated role partitions and their corresponding baseline roles. Features are extracted for facial, hairstyle, clothing, and body proportion partitions, and the average feature similarity of each effective partition is taken to obtain the average role identity consistency.

[0163] ;

[0164] In the formula, This indicates the consistency of average role identity; This indicates the number of evaluation samples with valid visible partitions; Indicates the first Feature similarity between each effective role partition and its corresponding role baseline partition. The value ranges from 0 to 1, with a larger value indicating better consistency in the character's identity.

[0165] In this experiment, the threshold values ​​for facial identity residuals were set to 0.18, hairstyle identity residuals to 0.22, clothing identity residuals to 0.24, and body proportion identity residuals to 0.15. These thresholds were determined based on manually judged acceptable role partitioning samples from preliminary experiments.

[0166] 2. Lens edge residual

[0167] Lens boundary residuals are used to evaluate the difference between the state of the character at the beginning of the target shot and the state of the character at the end of the preceding shot. The position residual, attitude residual, and motion residual are normalized and then weighted to obtain the lens boundary residuals:

[0168] ;

[0169] In the formula, Indicates lens edge residual; This represents the residual position of the character after alignment with the coordinate transformation relationships recorded by the state-dependent edges; This represents the posture residuals corresponding to key points of the human body and the body's orientation. This represents the motion residual corresponding to the direction and velocity of motion; , and These represent the weights of the position residual, attitude residual, and motion residual, respectively.

[0170] In this experiment, , and The values ​​were set to 0.30, 0.40, and 0.30 respectively, and the shot boundary residual threshold was set to 0.08. The smaller the shot boundary residual, the more continuous the transition between character states in adjacent shots.

[0171] 3. Character drift rate and boundary jump rate

[0172] Face drift rate is the proportion of shots with facial identity residuals exceeding the facial identity residual threshold out of all generated shots. Clothing drift rate is the proportion of shots with clothing identity residuals exceeding the clothing identity residual threshold out of all generated shots. Boundary jump rate is the proportion of shots with lens boundary residuals exceeding the lens boundary residual threshold out of all generated shots.

[0173] A single shot can simultaneously exhibit facial drift, clothing drift, and boundary jumps. Therefore, the number of these three types of abnormal shots is not added together, nor is the sum of the three considered as the total number of abnormal shots.

[0174] 4. First-time acceptance pass rate and final acceptance pass rate

[0175] The first-time pass rate represents the proportion of shots that pass the identity residual and boundary residual detection without needing to be regenerated or repaired after the initial generation. The final pass rate represents the proportion of shots that pass acceptance after the initial generation and a maximum of two repairs.

[0176] 5. Percentage of repaired area

[0177] The percentage of the repaired area is used to evaluate the amount of spatiotemporal data involved in anomaly handling:

[0178] ;

[0179] In the formula, Indicates the percentage of the repaired area; This indicates the total number of intra-frame pixels to be updated involved in this repair; This represents the total number of pixels in all video frames of the corresponding shot.

[0180] When the entire shot is regenerated, the repair area percentage is 1; when only the local character partition in the abnormal frame is updated, the repair area percentage is less than 1.

[0181] 6. Generation and processing time

[0182] The generation processing time refers to the graphics processor inference time consumed to obtain a final accepted shot, including the initial generation time and subsequent regeneration or partial repair time, but excluding manual annotation time and video file upload time. All processing times are recorded using the same timing interface.

[0183] Continuous indicators are represented by the mean and sample standard deviation of three repeated experiments; count indicators are represented by the number of shots after combining three repeated experiments and the corresponding proportion.

[0184] (v) Proportional Setting

[0185] The experimental verification process is as follows: Figure 5 As shown. The same short drama script, character baseline map and random seed were input into Example 1 and Comparative Examples 1 to 3, respectively, and the differences between the groups are as follows.

[0186] Comparative Example 1

[0187] Comparative Example 1 uses a single frontal character reference image to generate characters from all perspectives, without establishing a multi-view character anchor point set or a camera chain; each shot is generated independently. If a generated shot fails acceptance testing, the entire shot is regenerated.

[0188] Comparative Example 2

[0189] Comparative Example 2 uses multiple reference images from different perspectives for each character. One reference image is selected based on the overall perspective of the target shot, and its overall image features are used as the character generation condition. Comparative Example 2 inputs the overall image features of the last 8 frames of the preceding shot into the subsequent shot, but does not create a character partition-condition source map, nor does it distinguish the condition sources for facial features, hairstyle, clothing, and body proportions. If the generated shot fails acceptance, the entire shot is regenerated.

[0190] Comparative Example 3

[0191] Comparative Example 3 selects character anchor points according to character partitions and sets identity attention branches and temporal control branches, but does not create fixed conditional source snapshots, and does not distinguish between identity sources, boundary sources, and composite sources. When any residual exceeds the threshold, the entire shot is regenerated.

[0192] Example 1

[0193] Example 1 uses Figure 1 The complete method for steps S1 to S6 is as follows: First, a camera chain with state-dependent edges is established through the camera chain construction module; second, a multi-view character anchor point set is established through the character anchor point construction module; then, a character anchor point is uniquely selected for each target visible partition through the condition source binding module, and a character partition-condition source graph and a condition source snapshot are generated.

[0194] like Figure 3 As shown, the identity attention branch injects the identity features of the character anchor point into the target character partition according to the spatial gating mask, and the temporal control branch converts the character state at the end of the front-end camera into pose-motion features and then injects them into the video generation network.

[0195] like Figure 4 As shown, after generation, the identity residual detection unit and the boundary residual detection unit calculate the identity residual and the shot boundary residual, respectively, and the anomaly source determination unit determines the identity source, boundary source, or combined source. The repair feedback module forms a spatiotemporal repair map, and the local regeneration unit only updates the latent variables within the spatiotemporal repair map. After the shot passes acceptance, the terminal character state is extracted, and the terminal state fingerprint determines whether to pass a new state version to subsequent shots.

[0196] (vi) Overall Experimental Results

[0197] 1. Results of continuous indicators

[0198] The average character identity consistency, average shot boundary residual, repair area ratio, and average generation processing time for each group in three repeated experiments are shown in Table 1 below.

[0199]

[0200] Note: The boundary residual of Comparative Example 2 is 0.1113, which is expressed as 0.111 to three decimal places in the table; its sample standard deviation is 0.0025, which is expressed as 0.003 to three decimal places.

[0201] As shown in Table 1, the average character identity consistency in Example 1 is 0.921, which is 0.136 higher than that in Comparative Example 1 (0.785), representing an increase of 13.6 percentage points, or 17.32% on a relative change basis. This indicates that partitioning the multi-view baseline image of the character and uniquely binding a matching character anchor point to each target character partition can improve the preservation of the character's face, hairstyle, clothing, and body proportions under different viewpoints.

[0202] The average lens boundary residual of Example 1 is 0.050, which is 68.94% lower than that of Comparative Example 1 (0.161) and 54.95% lower than that of Comparative Example 2 (0.111). This shows that although using only the entire video frame at the end of the foreground lens as a reference can improve continuity, its effect is less than that of Example 1, which uses state-dependent edges to transmit the character's position, posture, and motion state.

[0203] In Example 1, the average repair area percentage was 0.210, meaning that on average, only 21.0% of the spatiotemporal region of the target shot needed to be updated per anomaly repair. Compared to regenerating the entire shot, the spatiotemporal region corresponding to the latent variables to be updated was reduced by 79.0%.

[0204] Example 1 yielded an average processing time of 21.2 GPU·min for a final accepted shot, a reduction of 37.65% compared to 34.0 GPU·min in Comparative Example 1, and a reduction of 22.63% compared to 27.4 GPU·min in Comparative Example 3. These results indicate that although Example 1 added condition source binding, residual attribution, and local repair processing, the computational savings from reducing the repetitive generation of the entire shot outweighed the computational costs of the added detection process.

[0205] 2. Results of count-type indicators

[0206] The experiment was repeated three times, and a total of 540 shots were obtained in each group. The count-type indexes are shown in Table 2 below.

[0207]

[0208] The above ratios are obtained by dividing the corresponding number of lenses by 540. For example, in Example 1, the facial drift rate is 27 divided by 540, which is 5.00%; the final acceptance rate is 525 divided by 540, which is 97.22%.

[0209] Compared to Comparative Example 1, Example 1 showed that the number of shots that passed the first acceptance test increased from 273 to 459, and the first acceptance pass rate increased by 34.44 percentage points; the number of shots that passed the final acceptance test increased from 411 to 525, and the final acceptance pass rate increased by 21.11 percentage points.

[0210] Comparative Example 2 improved character consistency and cross-camera continuity by using a multi-view overall reference map and overall frame information of the front-end camera. However, since the face, hairstyle and clothing sections of the same character cannot be adapted to anchor points separately, the face drift rate and clothing drift rate are still 17.22% and 20.00%, respectively.

[0211] Comparative Example 3 uses a partitioned anchor point and dual-branch generation method, which further reduces the facial drift rate, clothing drift rate and boundary jump rate. However, since the entire shot is regenerated after an anomaly occurs, the normal area is also recalculated, resulting in a final acceptance pass rate and average processing time that are still lower than those of Example 1.

[0212] Figure 6 The average shot boundary residual is used as the horizontal axis, and the average character identity consistency is used as the vertical axis. Comparative Example 1, Comparative Example 2, Comparative Example 3 and Example 1 are marked respectively.

[0213] The coordinates corresponding to Comparative Example 1 are (0.161, 0.785), Comparative Example 2 are (0.111, 0.834), Comparative Example 3 are (0.085, 0.887), and Example 1 is (0.050, 0.921). Example 1 is located at... Figure 6 The upper left region indicates that it has both low shot boundary residuals and high character identity consistency, rather than improving one metric at the expense of the other.

[0214] (vii) Ablation verification of core technical features

[0215] Example 2: Verification of Character Anchor Point Lag Switching

[0216] 120 adjacent character partitions containing continuous turns, gradually turning from the front to the side, were selected, and a total of 360 anchor point selection tasks were formed under 3 random seeds.

[0217] The first group uses a single switching threshold, which switches when the adaptation score of another candidate anchor point is higher than that of the current anchor point; the second group uses the retention threshold and switching threshold of the present invention, which switches only when the score of the current anchor point is lower than 0.62 and the score of another candidate anchor point is higher than 0.74.

[0218] Table 3 shows the results of character anchor point switching and appearance jump.

[0219]

[0220] As shown in Table 3, after adopting the retention threshold and switching threshold, the number of anchor point switching times decreased from 87 to 36, a reduction of 58.62%; the number of appearance jumps after switching decreased from 39 to 12, a reduction of 69.23%. This indicates that hysteresis switching not only reduces the number of data operations, but also reduces the jumps in facial contours, hairstyles, or clothing textures caused by repeated changes in character anchor points in continuous shots.

[0221] Example 3: Condition Source Snapshot Verification

[0222] We selected 150 shot repair tasks that had completed their initial generation but underwent character anchor point library updates before residual detection. Character anchor point library updates include replacing the baseline map, re-extracting partition features, or adjusting character partition boundaries.

[0223] When a conditional source snapshot is not used, the detection phase reselects character anchors from the updated character anchor library; when a conditional source snapshot is used, the detection and repair phases read the source map version, character anchor identifier, and predecessor state version saved at the time of generation.

[0224] Table 4 shows the impact of conditional source snapshots on source mismatch.

[0225]

[0226] The results in Table 4 above indicate that if the role anchor point is reselected during the detection phase, the generated result may be compared with a role anchor point that was not actually used during generation, leading to incorrect residual attribution. Fixing the condition source used for generation, detection, and repair by using a condition source snapshot can avoid source mismatch caused by asynchronous updates to the role anchor point library.

[0227] Example 4: Terminal Status Fingerprint Verification

[0228] 180 local repair events of the front-end camera were selected. Comparison revealed that 132 local repairs did not change the terminal character state, which consisted of the character's position, posture, direction of movement, speed of movement, and occlusion status; while 48 local repairs changed at least one of the terminal character states.

[0229] Without terminal status fingerprinting, the subsequent lens is marked as faulty whenever the front-end lens is repaired; with terminal status fingerprinting, the new status version is only transmitted to the subsequent lens when the terminal status fingerprint changes after repair.

[0230] Table 5 shows the impact of terminal status fingerprints on subsequent camera updates.

[0231]

[0232] Table 5 shows that after adopting terminal status fingerprinting, the number of subsequent shots marked as invalid decreased from 180 to 48, a reduction of 73.33%. This result indicates that by determining whether the local repair of the preceding shot actually changes the role state transmitted to the subsequent shot, it is possible to avoid reprocessing the subsequent shot simply because the background area of ​​the preceding shot or non-terminal frames have changed.

[0233] (viii) Typical local repair examples

[0234] Select a three-camera sequence that includes character A. Camera 1 shows character A walking to the right from the front; camera 2 shows character A continuing to walk from the right side; and camera 3 shows character A leaving the scene with their back to the camera.

[0235] according to Figure 1 As shown in step S1, the camera chain construction module establishes state dependency edges from camera one to camera two and from camera two to camera three. Following step S2, the character anchor point construction module establishes anchor points for the face, hairstyle, clothing, and body proportions from five multi-view reference images of character A.

[0236] according to Figure 3 As shown in the structure, when generating shot two, the condition source binding module binds the right-side facial anchor point to the right-side face partition, binds the three-quarters side hairstyle anchor point to the hairstyle partition, and binds the clothing anchor point to the clothing that can fully display the side pattern of the clothing. The identity attention branch injects the corresponding identity features according to the gating mask of each space. The timing control branch reads the posture—motion state—of character A walking to the right in the last 8 frames of shot one.

[0237] The initial generated results showed a change in the upper garment texture between frames 37 and 58. The clothing zone identity residual was 0.31, higher than the clothing identity residual threshold of 0.24; the lens boundary residual was 0.052, lower than the lens boundary residual threshold of 0.08. Therefore, the anomaly source determination unit identified this anomaly as the identity source.

[0238] according to Figure 4As shown in the process, the repair feedback module uses the clothing section of frame 37 as the initial repair area and propagates it to frame 58 via optical flow to form a spatiotemporal repair map. The spatiotemporal repair map covers 18.7% of the spatiotemporal region of the corresponding shot. The local regeneration unit calls the clothing anchor points recorded in the condition source snapshot, reuses the random seed, initial noise, and latent variables from the initial generation, and only updates the latent variables within the spatiotemporal repair map.

[0239] After the repair, the clothing identity residual decreased from 0.31 to 0.16, which is below the clothing identity residual threshold of 0.24; the lens boundary residual changed from 0.052 to 0.053, which is still below 0.08. This shows that the abnormal identity source was repaired without destroying the original lens boundary continuity.

[0240] This fix did not change the position, posture, or movement of the character A at the end of Scene 2. The terminal state fingerprints before and after the fix were the same, so no new state version was passed to Scene 3, and Scene 3 did not need to be regenerated.

[0241] The foregoing description of embodiments of the present invention, through which those skilled in the art are able to implement or use the present invention, will be readily apparent to those skilled in the art. Various modifications to these embodiments will be readily apparent to those skilled in the art. The general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novelty disclosed herein.

Claims

1. A short play generation method based on multi-view role anchor points and shot chain feedback, characterized in that, Includes the following steps: S1, parse the short drama script to obtain the shot nodes, and establish a shot chain with state-dependent edges according to the shared characters and shot connection relationship; S2, partition the multi-view baseline map of the character, calibrate the viewpoints and standardize the coordinates to form a set of character anchor points containing partition features and visibility confidence; S3, map the target character's pose to the standard coordinates, uniquely select the character anchor point for each target's visible partition, generate a character partition-condition source graph, and record the partition identifier, anchor point identifier, and spatial gating mask for each partition; S4, generate target shots along the lens chain, inject the identity features of the corresponding role anchor point into the video generation network through the identity attention branch according to the spatial gating mask, and inject the posture-motion features obtained by the transformation of the end role state of the foreground shot into the video generation network through the temporal control branch. After the end role state of the foreground shot associated with the target shot through the state dependency edge is transformed into posture-motion features, it is injected into the temporal control branch. S5, reverse map the character partition of the generated shot to the normal coordinates, compare it with the character anchor point recorded in the same partition to obtain the identity residual, compare the character state at the beginning of the target shot with the character state at the end of the foreground shot to obtain the boundary residual, and determine the first over-limit frame, abnormal partition and identity source, boundary source or composite source label. S6. Based on the tags, a spatiotemporal restoration map is formed. The identity source calls the role anchor point recorded in the abnormal partition, the boundary source calls the end role state of the preceding lens, and the composite source calls both simultaneously. In each denoising stage, only the hidden variables in the spatiotemporal restoration map are updated. After acceptance, the terminal role state is transmitted to the subsequent lens and the accepted lens is stitched together.

2. The short play generating method according to claim 1, characterized by, In step S1, the camera node records at least the character appearing, the target posture, the shooting angle, the character's movement direction, and the camera boundary; the state-dependent edge records the preceding camera identifier, the following camera identifier, the shared character identifier, the state item to be migrated, and the coordinate transformation parameters for converting the preceding camera coordinates to the following camera coordinates; a state version identifier is assigned to the end character state of the preceding camera, so that the subsequent camera generation and boundary residual calculation call the same end character state pointed to by the state version identifier.

3. The short play generating method according to claim 1, characterized by, In steps S2 and S3, a character specification template including facial, hairstyle, clothing and body proportion partitions is established, each character partition is mapped to the character specification template, and the view vector, partition identity features, partition visibility mask and specification coordinate mapping relationship are recorded for each character anchor point. Anchor point scores are generated based on viewpoint deviation, visibility confidence, and partition coverage. The same role anchor point is maintained in the visible partition of the same target in consecutive frames until its anchor point score is lower than the retention threshold and the anchor point score of another candidate anchor point is higher than the switching threshold. The switching threshold is higher than the retention threshold.

4. The short play generating method according to claim 3, characterized by, In steps S3 and S4, a source map version identifier is generated each time a character partition-condition source map is created or updated, and a condition source snapshot is saved when the target shot is generated. The condition source snapshot includes the source map version identifier and the partition identifier, anchor point identifier, spatial gating mask and normative coordinate mapping relationship corresponding to each target visible partition. The identity features of the partition pointed to by the anchor point are mapped to the key features and value features of the identity attention branch. Before the attention weight is normalized and before the attention score is normalized, the attention score corresponding to the spatial location outside the visible partition of the target is set to an invalid value according to the spatial gating mask.

5. The short play generating method according to claim 4, characterized by, In step S5, the generated shot is associated with the condition source snapshot saved when it is generated; for each generated partition, only its identity features are compared with the partition identity features pointed to by the anchor point identifier corresponding to the same partition identifier in the condition source snapshot, and other role anchor points are not reselected. The target camera's first character state is time-aligned with the front camera's last character state, as indicated by the state version identifier, to obtain the position residual, attitude residual, and motion residual, and the boundary residual is determined based on these three.

6. The short play generation method of claim 5, wherein, In steps S5 and S6, the frame that first exceeds the corresponding residual threshold is determined as the first over-limit frame according to the playback order of the shots; an identity source label is generated when only the identity residual exceeds the limit, a boundary source label is generated when only the boundary residual exceeds the limit, and a composite source label is generated when both the identity residual and the boundary residual exceed the limit; the source map version identifier, anchor point identifier, and status version identifier are written into the corresponding source label, and the consistency of the source map version identifier in the source label with the conditional source snapshot is checked before local regeneration.

7. The short play generating method according to claim 6, characterized by, In step S6, the abnormal partition in the first over-limit frame is used as the initial repair area, and the initial repair area is propagated along the time direction according to the optical flow of adjacent frames and the pose of the target character. After the consistency check is passed, the corresponding repair conditions are provided to the identity attention branch and the temporal control branch according to the source label. The initial noise when the target shot is first generated and the original latent variables of each denoising stage saved during the initial generation of the target shot are reused. In each denoising stage, the repaired latent variables are only written into the area within the spatiotemporal repair map. When the repaired identity residual and boundary residual do not exceed the limit, the acceptance is determined to be passed, and the subsequent shot is updated only when the state fingerprint of the terminal role changes.

8. A short play generation system based on multi-view character anchor and shot chain feedback, characterized in that, include: The shot chain construction module is used to parse the short drama script to obtain shot nodes and build a shot chain with state dependency edges according to the shared characters and shot connection relationship; Generate a state version identifier for the end role status of the accepted front-end camera, and associate the state version identifier with the corresponding state dependency edge; The character anchor point construction module is used to partition, calibrate, and standardize coordinates of the character's multi-view baseline map to establish a set of character anchor points. The Condition Source Binding module is used to map the target character's pose to the normalized coordinates, uniquely select the character anchor point for each target's visible partition, and generate a character partition-condition source graph that records the partition identifier, anchor point identifier, and spatial gating mask. The dual-branch generation module is used to inject the identity features of the corresponding character anchor point into the identity attention branch according to the spatial gating mask, and to inject the character state at the end of the foreground shot into the temporal control branch after converting it into posture-motion features, so as to generate the target shot. The residual attribution module is used to calculate the identity residual based on the character anchor points recorded in the same character partition, calculate the boundary residual based on the character state at the end of the forward camera pointed to by the state dependency edge, and determine the first over-limit frame, abnormal partition and source label. The repair feedback module is used to form a spatiotemporal repair map based on the source tags. Based on the identity source tags, boundary source tags, or composite source tags, it calls the character anchor point, the end character state of the preceding shot, or a combination of the two as local regeneration conditions to implement local regeneration, and feeds back the accepted terminal character state to the subsequent shot.

9. The skit generation system of claim 8, wherein, The conditional source binding module includes an anchor point scoring unit, an anchor point locking unit, and a source snapshot unit. The anchor point scoring unit is used to generate anchor point scores based on viewpoint deviation, visibility confidence, and partition coverage. The anchor point locking unit is used to control the retention or switching of anchor points within consecutive visible frames based on retention thresholds and switching thresholds. The source snapshot unit is used to save the source map version identifier, partition identifier, anchor point identifier, spatial gating mask, and canonical coordinate mapping relationship used when generating the target shot.

10. The skit generation system of claim 9, wherein, The residual attribution module is used to associate the generated shot with the condition source snapshot saved at the time of its generation, using the character anchor point recorded in the same partition of the condition source snapshot as the identity residual comparison benchmark, and using the character state at the end of the preceding shot pointed to by the state version identifier as the boundary residual comparison benchmark; the repair feedback module includes a source consistency verification unit, a condition routing unit, a latent variable update unit, and a state propagation unit, which are respectively used to verify the source map version identifier, provide repair conditions according to the source label, update only the latent variables in the spatiotemporal repair map, and control the update of subsequent shots according to the state fingerprint changes of the terminal character state.

Citation Information

Patent Citations

  • Scene change controllable text video method, system and equipment

    CN119583907A

  • Method for obtaining frame sequence from text and related device

    CN120279470A