Video generation method and device, electronic equipment and medium

Abstract

The embodiment of the invention discloses a video generation method and device, electronic equipment and a medium, and relates to the technical field of artificial intelligence, one specific implementation mode of the method comprises the steps that multi-dimensional video data corresponding to an original video is acquired, and the multi-dimensional video data is used for representing full-amount structured information related to a story, a role and a scene in the original video; constructing a content generation model adaptive to the original video based on the multi-dimensional video data; and based on the content generation model and a content generation request input by the user, generating expanded content data conforming to the original video logic, so as to generate a target video according to the expanded content data. According to the method, the manual investment and workload in the drama supplementary creation can be greatly reduced, the integrating degree of the expanded content and the original created drama is guaranteed by fusing the multi-dimensional data, and the drama creation efficiency is effectively improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a video generation method, apparatus, electronic device, and medium. Background Technology

[0002] Currently, in video-based storyline supplementation and generation scenarios, the relevant technologies mainly rely on user-inputted creative requirement text. By extracting multiple thematic summaries from the text, and then expanding the thematic summaries in multiple rounds, the storyline content is finally generated based on the expansion results.

[0003] However, the relevant technologies have obvious drawbacks: on the one hand, they require a high degree of human intervention, especially in video plot analysis and summary extraction, which still require a lot of human intervention, resulting in low creation efficiency and reliance on personal experience; on the other hand, multiple content expansions based on summaries can easily lead to the generated content gradually deviating from the original video or text theme, and it is difficult to effectively control the consistency between plot logic and character behavior, affecting the fit between the generated content and the original video. Summary of the Invention

[0004] This application provides a video generation method, apparatus, electronic device, and medium to solve the problem in related technologies that video plot supplementation content relies on manual labor, resulting in low video generation efficiency and easy deviation from the consistency of the original video theme and character behavior logic.

[0005] In a first aspect, embodiments of this application provide a method for video generation, the method comprising: acquiring multi-dimensional video data corresponding to an original video, wherein the multi-dimensional video data is used to represent all structured information related to the plot, characters, and scenes in the original video; constructing a content generation model adapted to the original video based on the multi-dimensional video data; and generating extended content data that conforms to the logic of the original video based on the content generation model and a content generation request input by a user, so as to generate a target video based on the extended content data.

[0006] In some embodiments, the multi-dimensional video data includes video plot data. Obtaining the multi-dimensional video data corresponding to the original video includes: dividing the original video into at least one video segment to be processed using a preset segmentation algorithm; extracting keyframes from the first video segment to be processed in the at least one video segment to be processed to obtain a first keyframe set of the first video segment to be processed; performing speech recognition on the video audio corresponding to the first keyframe set to obtain a first audio text; determining the plot node corresponding to the first keyframe set based on the first audio text and the first subtitle text of the first keyframe set; and determining the video plot data based on the plot node.

[0007] In some embodiments, the multi-dimensional video data includes video character data, which includes at least one of character appearance data, character personality data, and character behavior data. Obtaining the multi-dimensional video data corresponding to the original video includes: dividing the original video into at least one video segment to be processed using a preset segmentation algorithm; extracting keyframes from a first video segment to be processed in the at least one video segment to be processed to obtain a first set of keyframes for the first video segment to be processed; performing character detection on the keyframes in the first set of keyframes to obtain a first character in the first set of keyframes; extracting the character appearance data of the first character using image recognition technology, which includes at least one of clothing data, hairstyle data, and facial feature data; extracting the character text corresponding to the first character, and determining the character personality data of the first character based on the character text and sentiment analysis technology, where the character text includes at least one of dialogue text and subtitle text; and obtaining the joint coordinate information of the target character using human pose estimation technology, and determining the character behavior data of the first character based on the joint coordinate information and the character text.

[0008] In some embodiments, multi-dimensional video data includes video scene data, which includes at least one of direct scene data and indirect scene data. Obtaining multi-dimensional video data corresponding to the original video includes: dividing the original video into at least one video segment to be processed using a preset segmentation algorithm; extracting keyframes from the first video segment to be processed in the at least one video segment to be processed to obtain a first keyframe set for the first video segment to be processed; performing scene recognition on the keyframes in the first keyframe set to determine the direct scene data of the first keyframe set, which includes at least one of scene type and scene location information; and inferring the indirect scene data of the first keyframe set based on the character text of the first character in the first keyframe set and the video plot data corresponding to the first keyframe set.

[0009] In some embodiments, generating extended content data that conforms to the original video logic based on the content generation model and the user-inputted content generation request includes: parsing the content generation request to determine the content generation requirements and contextual constraints that adapt to the content generation requirements; generating at least one candidate content element that conforms to the original video logic based on the contextual constraints, content generation requirements, and content generation model; responding to the user's selection operation on at least one candidate content element to determine the target content element selected by the user; and generating extended content data based on the target content element and multi-dimensional video data.

[0010] In some embodiments, generating a target video based on extended content data includes: generating a supplementary content video based on the extended content data and a preset character model; merging the supplementary content video with the original video to obtain a target video when the content generation requirement is to fill in the gaps in the original video's plot; and determining the supplementary content video as the target video when the content generation requirement is to generate a video independently.

[0011] Secondly, embodiments of this application provide a video generation apparatus, the apparatus comprising: The acquisition unit is used to acquire multi-dimensional video data corresponding to the original video. The multi-dimensional video data is used to represent the full amount of structured information related to the plot, characters, and scenes in the original video. The building unit is used to construct a content generation model adapted to the original video based on multi-dimensional video data; The generation unit is used to generate extended content data that conforms to the original video logic based on the content generation model and the content generation request input by the user, so as to generate the target video based on the extended content data.

[0012] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor, wherein, when the processor runs the computer program, it performs the method described in any embodiment of the first aspect.

[0013] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods described in any embodiment of the first aspect.

[0014] Fifthly, embodiments of this application provide a computer program product including a computer program that, when executed by a processor, performs the method described in any embodiment of the first aspect.

[0015] This application provides a video generation method that acquires multi-dimensional video data corresponding to the original video. This multi-dimensional video data represents all structured information related to the plot, characters, and scenes in the original video. Based on the multi-dimensional video data, a content generation model adapted to the original video is constructed. Based on the content generation model and the user-inputted content generation request, extended content data conforming to the logic of the original video is generated, and a target video is generated based on this extended content data. This method significantly reduces the manual input and workload in creating supplementary plots. Furthermore, by integrating key information such as plot development, multi-dimensional character characteristics, and scene settings, it ensures that the generated extended content closely matches the original work in terms of theme, character consistency, and plot coherence, thereby effectively improving plot creation efficiency and guaranteeing the fit between the extended content and the original plot.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are merely embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort, and this application can be applied to other similar scenarios based on the provided drawings.

[0018] Figure 1 A flowchart illustrating a video generation method provided in this application embodiment; Figure 2 A flowchart illustrating the second video generation method provided in this application embodiment; Figure 3 A flowchart illustrating the third video generation method provided in this application embodiment; Figure 4 A schematic diagram illustrating the process of constructing a content generation model as provided in an embodiment of this application; Figure 5 A general flowchart of an exemplary video generation method provided for embodiments of this application; Figure 6 This is a schematic diagram of the structure of a video generation device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. The described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] It should be noted that the terms "system," "device," "unit," and / or "module" used in this application are methods of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they can be replaced by other expressions.

[0021] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0022] When watching entertainment videos, users often encounter two scenarios: first, the camera mainly focuses on the main characters, while the experiences and growth of some supporting characters are glossed over; second, due to the length limitations of the series, some of the main characters' experiences are shortened or replaced by narration. In response to these situations, users typically engage in secondary creation of the original content to enrich the plot and create personalized derivative works.

[0023] Currently, in video-based storyline supplementation and generation scenarios, the relevant technologies mainly rely on user-inputted creative requirement text. First, natural language processing technology is used to analyze the core requirements of the text, and then multiple independent topic summaries are extracted from it. Subsequently, multiple rounds of progressive content expansion are carried out based on each topic summary. Finally, based on the text materials formed by multiple rounds of expansion, the corresponding storyline content is generated by combining simple image or video splicing technology.

[0024] However, the relevant technologies have obvious drawbacks: on the one hand, they require a high degree of human intervention, especially in video plot analysis and summary extraction, which still require a lot of human intervention, resulting in low creation efficiency and reliance on personal experience; on the other hand, multiple content expansions based on summaries can easily lead to the generated content gradually deviating from the original video or text theme, and it is difficult to effectively control the consistency between plot logic and character behavior, affecting the fit between the generated content and the original video.

[0025] To address the problems in existing technologies, this application proposes a video generation method. By acquiring full-volume structured multi-dimensional video data related to the plot, characters, and scenes in the original video, a content generation model adapted to the original video is constructed. Combined with user content generation requests, extended content data that conforms to the original creative logic is generated. This significantly reduces the manual input and workload in supplementary creation. Furthermore, by deeply replicating the original creative plot, character characteristics, and scene settings through multi-dimensional data, it effectively avoids the problems of extended content deviating from the theme of the original work and inconsistent character behavior. This significantly improves creative efficiency and the fit between extended content and the original creative plot.

[0026] The following section provides a detailed description of a video generation method provided in this application, with reference to the accompanying drawings.

[0027] Figure 1 A flowchart of a video generation method provided in an embodiment of this application is shown. Figure 1 As shown, the video generation method includes steps 101-103.

[0028] Step 101: Obtain multi-dimensional video data corresponding to the original video. Multi-dimensional video data is used to represent the full amount of structured information related to the plot, characters, and scenes in the original video.

[0029] In the embodiments of this application, multi-dimensional video data includes at least one of video plot data, video character data, and video scene data. Structured information refers to converting unstructured video footage, audio, and subtitles into tagged data that can be read by a computer, such as a list of plot nodes, character feature parameters, and scene classification tags.

[0030] To aid in better understanding, this application uses an original video consisting of 12 episodes, each 30 minutes long, featuring protagonist A, supporting character B, and passerby C, as an example to explain the video generation method provided in this application.

[0031] Specifically, this application utilizes multimodal techniques to comprehensively analyze the original video: for example, it employs shot segmentation algorithms to break down video scenes and extract keyframes to obtain visual information; it extracts textual information from the video through speech recognition and subtitle analysis, and combines Natural Language Processing (NLP) techniques to identify plot points; it uses computer vision techniques such as object detection and pose estimation to extract the appearance and behavioral data of characters in the original video; and it constructs a scene-related structured list through image classification and semantic reasoning. All of the above data are integrated to form a unified, multi-dimensional video data set.

[0032] Step 102: Based on multi-dimensional video data, construct a content generation model adapted to the original video.

[0033] In the embodiments of this application, the content generation model is a multimodal fusion model constructed in this application, which has the ability to generate plot, character actions and scene images based on the logic of the original work. It is the core carrier for generating fan content (i.e. fan derivative content, which is a work created by secondary creation based on the characters, worldview and plot framework of the original original cultural and entertainment works).

[0034] This application employs a multimodal fusion architecture, combining Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and Transformer models. Specifically, for image data (keyframes), CNNs (such as ResNet and VGG (Visual Geometry Group)) are used for feature extraction to capture visual information in the images. For speech and caption text data, a Transformer model is first used for semantic understanding and feature extraction, and then LSTM is used to process the time-series information of the text to capture contextual semantic relationships. Finally, the features from images, speech, and text are fused and input into a fully connected layer for the final prediction or classification task.

[0035] In one example, the specific construction process of the content generation model is as follows: First, the multi-dimensional video data is preprocessed: The plot data undergoes temporal encoding, arranging plot nodes in the original video playback order, labeling the causal relationships between nodes, and transforming it into a temporal feature sequence recognizable by the model. Character data is tagged, converting character appearance parameters, personality tags, and behavioral logic into structured tags and binding them to character identifiers. Scene data is categorized and encoded, classifying scenes by type and location, generating scene feature codes, and establishing a mapping between scenes and plot nodes.

[0036] Next, model training and parameter optimization are performed: First, the training task and loss function are determined. For sub-tasks such as plot generation, character behavior prediction, and scene matching, the cross-entropy loss function is used for classification tasks, and the mean squared error loss function is used for regression tasks. Then, a training optimization strategy is configured, using an optimizer (such as the Adam optimizer) and setting an initial learning rate (e.g., 0.001). Every n iterations (e.g., 50 iterations), the learning rate is iterated and multiplied by 0.9 to balance the rapid convergence in the early stages of training with the fine-tuning in the later stages. An early stopping strategy is adopted: if the validation set loss no longer decreases after m consecutive iterations (e.g., 10 iterations), training is stopped to avoid overfitting. Finally, preprocessed multi-dimensional video data is input for training, allowing the model to learn the plot logic, character behavior patterns, and scene and plot matching rules of the original video. During training, data augmentation operations such as random flipping, rotation, and scaling are performed on the image data to increase data diversity and improve the model's generalization ability. Synonym replacement, random insertion, and deletion operations are performed on the text data to expand the text data sample.

[0037] Ultimately, the trained content generation model can not only understand the plot logic and character design of the original work, but also call on the supporting material model to generate visual content that conforms to the style of the original work.

[0038] Step 103: Based on the content generation model and the content generation request input by the user, generate extended content data that conforms to the original video logic, and generate the target video based on the extended content data.

[0039] In the embodiments of this application, content generation requirements refer to creative requests made by users in natural language or through interactive selection (such as clicking on plot nodes), for example: "I want to see what happened to supporting character B alone in the training ground between episode 1 and episode 2." Extended content data refers to structured content descriptions generated by the content generation model to constitute new videos. This can be scripts containing storyboards, dialogues, and action descriptions, or it can be more fundamental data such as scene and character action sequences.

[0040] Specifically, the process first parses the user's content generation request and extracts the core requirements, such as "supplementing the side story of supporting character B in episodes 1 and 2 of the video." Then, it calls the content generation model to generate multiple candidate plot schemes based on plot nodes, character characteristics, and scene lists from multi-dimensional video data. Next, it receives the user's selection of candidate schemes and further refines and generates extended content data such as storyboards, character action sequences, and scene layouts. Finally, based on the extended content data, combined with pre-built character models and animation generation technology, the target video is rendered.

[0041] It is understandable that the content generation model constructed in this application is not limited to video creation, but can also be applied to multiple fields such as text-based fan fiction creation, comic storyboard generation, and audio drama production.

[0042] In summary, the video generation method proposed in this application involves obtaining multi-dimensional video data corresponding to the original video. This multi-dimensional video data represents all structured information related to the plot, characters, and scenes in the original video. Based on the multi-dimensional video data, a content generation model adapted to the original video is constructed. Based on the content generation model and the user-inputted content generation request, extended content data conforming to the logic of the original video is generated, and the target video is generated based on the extended content data. This significantly reduces the manual input and workload in plot supplementation creation. By integrating key information such as plot development, multi-dimensional character characteristics, and scene settings, it ensures that the generated extended content is highly consistent with the original work in terms of theme, character consistency, and plot coherence, thereby effectively improving plot creation efficiency and guaranteeing the fit between the extended content and the original plot.

[0043] As one possible implementation method, Figure 2 The flowchart of the second video generation method is shown. Based on the above embodiment, it obtains multi-dimensional video data corresponding to the original video, including the following steps: Step 201: Divide the original video into at least one video segment to be processed using a preset segmentation algorithm.

[0044] In the embodiments of this application, multi-dimensional video data includes video plot data, video character data (including at least one of character appearance data, character personality data, and character behavior data), and video scene data (including at least one of direct scene data and indirect scene data).

[0045] The preset segmentation algorithm is a pre-defined shot segmentation algorithm that can divide the original video into multiple shots. It can be selected according to actual needs and is not limited in the embodiments of this application. This application takes the threshold method based on color histogram difference as an example.

[0046] Specifically, the color histogram difference between two adjacent frames of the original video is first calculated. When the difference value is greater than a preset threshold (such as 0.8, which is an empirical value and can be adjusted according to the complexity of the video content), it is determined to be a shot switching point. Based on the shot switching point, the original video is divided into several video segments to be processed, and each segment corresponds to a complete and independent scene shot.

[0047] Step 202: Extract keyframes from the first video segment to be processed in at least one video segment to be processed, and obtain the first keyframe set of the first video segment to be processed.

[0048] In the embodiments of this application, the first video segment to be processed is any sample segment among the video segments to be processed, used to exemplify the extraction process. A keyframe refers to one or more frames of images that can represent the core visual content and information of a shot.

[0049] Specifically, this application adopts a uniform sampling strategy, sampling the first video segment to be processed at a preset frequency (e.g., extracting 1 frame every 5 frames); and performs quality screening on the sampled keyframe images to remove blurry or duplicate frames; and integrates the screened frame images to form the first keyframe set.

[0050] Step 203: Perform speech recognition on the video speech corresponding to the first keyframe set to obtain the first speech text.

[0051] In this embodiment of the application, the video audio corresponding to the first keyframe set is the audio data synchronized with the first video segment to be processed.

[0052] Specifically, firstly, the audio data corresponding to the first video segment to be processed is extracted, and the sampling rate is set (e.g., 16000Hz); then, the speech recognition interface (e.g., Baidu Speech Recognition API (Application Programming Interface)) is called, the Chinese recognition mode is selected, and the audio data is converted into text; the recognition results are proofread and cleaned to remove invalid words and obtain the standardized first speech text.

[0053] Step 204: Based on the first voice text and the first subtitle text of the first keyframe set, determine the plot node corresponding to the first keyframe set.

[0054] In the embodiments of this application, the first subtitle text is the subtitle text content that comes with the first video segment to be processed. Plot nodes are text tags that represent plot twists or key information (such as "protagonist A obtains a key prop" or "supporting character B has a dispute with protagonist A"), marking a relatively complete story point of situational change, occurrence of important events, or change of character status.

[0055] Specifically, the first audio text and the first subtitle text are fused to form a complete text dataset. A pre-trained language model (such as the BERT model, which is a pre-trained language model that uses a Transformer encoder and is trained through bidirectional context to obtain deep semantic representations) is used to perform semantic analysis on the text dataset and calculate the semantic similarity between sentences. When the similarity is lower than a preset threshold (e.g., 0.7), it is determined as a new plot node. The identified plot nodes are then labeled to clarify the keyframes and core events corresponding to the plot nodes.

[0056] Step 205: Determine video plot data based on plot nodes.

[0057] In the embodiments of this application, the video plot data is a structured plot outline formed by arranging all plot nodes in chronological order, including information such as node events, occurrence order, and associated characters.

[0058] Specifically, the plot nodes of all the video clips to be processed are integrated and sorted according to the playback order of the original video. No attribute tags are added to each plot node, including the corresponding clip, keyframe position, core event description, related characters, etc. The sorted set of tagged plot nodes is determined as the video plot data.

[0059] Step 206: Perform character detection on the keyframes in the first keyframe set to obtain the first character in the first keyframe set.

[0060] In the embodiments of this application, the first character refers to all characters detected from the first keyframe set, such as protagonist A, supporting character B, etc., and the specific number can be one or more. Character detection refers to the technology of identifying and locating characters from keyframe images.

[0061] Specifically, this application calls a person detection model (such as the YOLOv5 model) and sets a confidence threshold (for example, setting the confidence threshold to 0.5 to ensure accurate detection of characters) to detect each frame in the first keyframe set, identify the human targets in the image, and output the bounding box coordinates of the characters; classifies and labels the detected characters to determine their identities (for example, matching the detected characters with a pre-established "work character library" (which can be initialized from the cast and crew list, posters, etc.) to determine their identities as "character A", "character B" or "passerby C"), thus forming a first character list.

[0062] Step 207: Extract the character appearance data of the first character using image recognition technology. The character appearance data includes at least one of clothing data, hairstyle data, and facial feature data.

[0063] In the embodiments of this application, character appearance data are structured parameters that characterize the character's external image and form the basis of character modeling. Clothing data, hairstyle data, and facial feature data correspond to the quantitative data of the character's clothing color and style, hairstyle, and facial features, respectively.

[0064] Specifically, this application utilizes image recognition technology (such as the OpenCV library) to segment the bounding box region of the character and extract independent images of the character. The segmented images are converted to the HSV color space, and the color distribution of the clothing area is statistically analyzed to determine the clothing data. A deep learning model (such as the HairNet model, which has an accuracy rate of over 90%) is called to identify the character's hairstyle and generate hairstyle data. Facial feature extraction algorithms are used to identify the coordinates of key facial features, forming facial feature data. The above data are integrated to construct the character's appearance data for the first character.

[0065] Step 208: Extract the character text corresponding to the first character, and determine the character personality data of the first character based on the character text and sentiment analysis technology. The character text includes at least one of the dialogue text and subtitle text.

[0066] In the embodiments of this application, the character text refers to the lines and subtitles corresponding to the first character in the first video segment to be processed. The character personality data is the character personality tags obtained through text analysis, such as "calm," "impulsive," and "scheming."

[0067] Specifically, dialogue content related to the first character is extracted from the first audio text and the first subtitle text to form character text; an artificial intelligence model is called, and prompts such as "analyze the personality traits reflected in the character's dialogue" are input to perform sentiment analysis and semantic interpretation on the character text; based on the model output, character personality tags are generated to obtain character personality data.

[0068] Step 209: Using human pose estimation technology, obtain the joint coordinate information of the target character, and determine the character behavior data of the first character based on the joint coordinate information and the character text.

[0069] In the embodiments of this application, human pose estimation technology refers to the technology of identifying the positions of human joints in an image, used to analyze character movements. Joint coordinate information refers to the two-dimensional or three-dimensional coordinates of each joint of the character's body. Character behavior data is structured information used to characterize the character's actions and intentions, such as "walking," "waving," and "arguing." Character behavior information includes character actions and action intention information.

[0070] Optionally, human pose estimation is performed on the first character to determine the character's actions, and the action intention information of the character's actions is determined based on the character's actions, character text, and video plot data.

[0071] In one example, a human pose estimation model (such as the OpenPose model) is invoked, with the input image size set to 368×368. Using an 18-joint model from the COCO dataset, the pose of the character in the first keyframe is estimated, outputting the character's joint coordinates to form a pose sequence. The joint coordinates of 10 consecutive frames are then input into an LSTM-based action recognition model to identify the character's action type. Combined with the context of the character's dialogue, the intention behind the action is interpreted. Integrating the action type and intention yields the character's behavior data for the first character.

[0072] Step 210: Perform scene recognition on the keyframes in the first keyframe set to determine the direct scene data of the first keyframe set. The direct scene data includes at least one of scene type and scene location information.

[0073] In the embodiments of this application, direct scene data refers to scene information obtained through direct image recognition.

[0074] Specifically, an image classification model (such as the ResNet50 model) is invoked to classify the frame images in the first keyframe set and determine the scene type, such as indoor, outdoor, or street. For frame images containing landmark features, landmark recognition technology (such as using the Baidu Maps API) is employed to match the image landmark features with the map database to obtain specific scene location information. The scene type and location information are then integrated to form direct scene data.

[0075] Step 211: Based on the character text of the first character in the first keyframe set and the video plot data corresponding to the first keyframe set, infer the indirect scene data of the first keyframe set.

[0076] In the embodiments of this application, indirect scene data refers to scene information that cannot be directly identified through images and needs to be obtained by combining text and plot reasoning, such as "character A's childhood hometown in his memories".

[0077] Specifically, we first extract descriptive words about the scene from the character's text, such as "hometown" and "the beach last summer." Combined with the video plot data corresponding to the segment, we analyze the time and causal relationships of the plot. Through NLP semantic reasoning technology, we infer scene information that does not appear directly in the scene and label the reasoning results to form indirect scene data.

[0078] It should be noted that annotation can be done with the help of a semi-automatic annotation tool.

[0079] In summary, this application utilizes multimodal technologies such as shot segmentation, keyframe extraction, speech recognition, and character detection to achieve comprehensive analysis of the original video from visual to textual perspectives. It constructs multi-dimensional video data encompassing plot, characters, and scenes, providing a precise and comprehensive data source for the construction of content generation models. This effectively avoids issues such as the generated extended content deviating from the theme of the original work and inconsistencies in character behavior, further improving creation efficiency and the fit between the extended content and the original work.

[0080] As one possible implementation method, Figure 3 The flowchart of the third video generation method is shown. Based on the above embodiments, it generates extended content data that conforms to the original video logic based on the content generation model and the user's input content generation request, including the following steps: Step 301: Parse the content generation request and determine the content generation requirements and contextual constraints that adapt to the content generation requirements.

[0081] In the embodiments of this application, content generation requirements refer to the user's core creative needs, such as "supplementing the side story of supporting character B in episodes 1 and 2" or "generating a short video from the perspective of character C". Contextual constraints include at least one of the following: the scope of plot nodes in the original video, the target character, and the scene type.

[0082] Specifically, the system first receives natural language requests or visual selection instructions from users, then uses NLP parsing technology to extract the core elements of the requests and determine the content generation requirements. Based on multi-dimensional video data, it automatically generates corresponding contextual constraints, such as the plot not exceeding the timeline from episode 1 to episode 2, character B's behavior needing to conform to their "calm" personality label, and scenes needing to be selected from a pre-built list of scenes.

[0083] Step 302: Based on contextual constraints, content generation requirements, and content generation model, generate at least one candidate content element that conforms to the original video logic.

[0084] In the embodiments of this application, the candidate content element is a plot scheme generated by the content generation model that meets the requirements and constraints, and may include at least one of candidate plots, candidate scenes, and candidate character actions.

[0085] Specifically, the contextual constraints and content generation requirements are input into the content generation model. Based on the learned original plot logic and character settings, the model generates multiple different candidate plot schemes and scores each candidate scheme. The schemes are then sorted from highest to lowest according to their relevance to the original work, forming a list of candidate content elements.

[0086] Step 303: In response to the user's selection operation on at least one candidate content element, determine the target content element selected by the user.

[0087] In this embodiment of the application, the target content element is the core plot scheme that the user finally selects for generating extended content.

[0088] Specifically, a list of candidate content elements is displayed to users through a visual interface. Each element comes with a plot description and a fit rating. Users can choose the candidate options they prefer as needed. After receiving the user's selection instructions, the system supports users in modifying and combining the candidate elements.

[0089] Step 304: Generate extended content data based on the target content elements and multi-dimensional video data.

[0090] In the embodiments of this application, the extended content data is structured creative data obtained by refining the target content elements, which may include at least one of storyboard descriptions, character action details, and scene configuration information.

[0091] Specifically, taking the target content elements as the core, and combining the character's appearance, personality, and behavior data from multi-dimensional video data, the details of the character's actions and dialogue are refined. The corresponding scene data is matched to determine the scene location and environmental characteristics where the plot takes place. A detailed plot outline and storyboard are generated, clarifying the parameters such as the screen content, character position, and shot duration of each storyboard. The above information is integrated to form complete extended content data.

[0092] Finally, the specific process for generating the target video based on the extended content data in this application is as follows: 1) Generate supplementary content videos based on expanded content data and preset role models; In the embodiments of this application, the preset character model is a character visual model pre-created based on multi-dimensional data of the original video. It is divided into two categories: two-dimensional (2D) face-swapping models (such as GAN network face-swapping technology) and three-dimensional (3D) modeling models (such as 3D modeling based on multi-view reconstruction). Specifically, regarding the training parameter settings, for 2D models (taking GAN as an example), the learning rate is generally set between 0.0001 and 0.001, the batch size commonly takes values ​​of 16 or 32, and the number of training iterations varies from hundreds to thousands depending on the dataset size and task difficulty. For 3D models (taking deep learning-based multi-view stereo network as an example), during training, the loss function is constructed based on the difference between the reconstructed model and the real model, the number of iterations is approximately 1500-3500, and the learning rate is between 0.0003 and 0.0008.

[0093] In one example, a pre-made preset character model is first called. If it is a 2D video, the GAN network face-swapping model is enabled to transfer the facial features of the character in the original video to the basic model in the material library to ensure that the character's facial features and expression are consistent with the original. If it is a 3D video or content that needs to be presented in stereo, the multi-view reconstruction 3D modeling model is enabled. The three-dimensional structure of the character is restored based on the multi-view images of key frames, matching the height, body shape, and clothing details of the original.

[0094] Next, the storyboard scripts in the expanded content data are analyzed to extract parameters such as the screen content, character positions, action instructions, and scene information for each storyboard. Then, based on the character behavior data and preset character models in the expanded content data, skeletal animation technology is used to drive the character models to complete specified actions, with the range and rhythm of the actions matching the character's personality (e.g., calm characters move slowly, while impulsive characters move quickly).

[0095] Then, it calls the direct or indirect scene data from the multi-dimensional video data to render the corresponding scene background and match the lighting and perspective required by the storyboard (such as the lighting effects of indoor scenes and the depth of field settings of outdoor scenes). Based on the dialogue text in the extended content data, it matches the corresponding timbre and tone parameters with the character's personality tags to generate dubbing audio; at the same time, it adds scene sound effects (such as ambient sounds and action sound effects) to synchronize with the visuals.

[0096] Finally, the motion vectors between keyframes of the storyboard are calculated using the optical flow algorithm, and transition frames are added to ensure that character movements and camera transitions are smooth and natural, ultimately integrating them to form a complete supplementary content video.

[0097] 2) When the content generation requirement is to fill in the gaps in the original video's plot, the supplementary content video is merged with the original video to obtain the target video; In the embodiments of this application, the plot gap refers to the interval of plot segments that are not fully shown in the original video, such as the time gap between two episodes, the side plot period outside the protagonist's perspective, etc.

[0098] In one example, the process begins by locating the gaps in the storyline and unifying the resolution, frame rate, and encoding format of the supplementary content video with the original video. Then, by comparing the character appearances, scene styles, and color tone parameters of the supplementary content video with the original video using a model, the color and brightness of the supplementary content video are fine-tuned to match the visual style of the original work. Finally, the video streams are spliced ​​together in the order of "beginning of the original video - supplementary content video - end of the original video" to generate the complete target video.

[0099] 3) If the content generation requirement is to generate an independent video, the supplementary content video will be designated as the target video.

[0100] In embodiments of this application, it is also possible to generate complete fan-made video works based solely on user needs, without relying on original video splicing, such as independent side stories from the perspective of supporting characters or entirely new short videos within the original work's world view. In this case, the generated supplementary content videos can be directly optimized in post-production, such as adjusting frame rate, resolution, and color parameters, to meet the user's fan-made creation needs.

[0101] In summary, this application constructs a content generation model based on the full amount of structured, multi-dimensional video data related to the plot, characters, and scenes in the original video. Combined with user content generation requests, it can generate extended content data that conforms to the logic of the original work. This not only significantly reduces the manual input and workload of fan creation, but also significantly improves the efficiency of video creation and the fit between the extended content and the original work.

[0102] In one example, to help better understand the video generation scheme provided in this application, such as Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the process of constructing a content generation model, as provided in an embodiment of this application.

[0103] Reference Figure 4First, the original video footage undergoes compliance and integrity checks. After standardization, it enters the data preprocessing stage, extracting keyframes through shot segmentation, image similarity analysis, and uniform sampling. Then, object annotation (marking characters, object categories, and boundaries), action annotation (marking character action types and start / end frame numbers), and scene annotation (clarifying scene types and attributes) are performed. Simultaneously, character or object classification for complex scenes is completed. The training data is enhanced through image flipping / rotation, video speed adjustment / cropping, and other operations. In the model selection and construction phase, corresponding models are matched for different data types (e.g., using CNN to extract visual features, RNN (Recurrent Neural Network) / LSTM to handle temporal relationships, and Transformer to parse text semantics; ultimately, a multimodal fusion model integrates all features). Simultaneously, a model architecture adapted to the task is designed (e.g., CNN-LSTM cascade for action recognition), and training parameters such as the Adam optimizer and cross-entropy loss function are configured. After model training, the model performance is evaluated through tasks such as classification and regression. Based on the evaluation results, model parameters are optimized, and finally, a content generation model adapted to the original video is output.

[0104] This application also provides a specific example to further explain the video generation method provided in this application, such as... Figure 5 As shown, Figure 5 This is a general flowchart of an exemplary video generation method provided in an embodiment of this application.

[0105] Reference Figure 5 The scenario is set against the backdrop of a user wanting to create a secondary narrative between video plot nodes V1 and V2 (from the perspective of supporting character B). After the content generation model is trained, the user inputs a content generation request: "Create a secondary narrative between plot nodes V1 and V2, from the perspective of supporting character B." Based on the plot nodes already in the database (including multi-dimensional video data such as episode number, major events, and scenes), the user can filter nodes in two ways: manually select V1 and V2 plot nodes in the management platform's visual interface, or trigger the AI ​​system to filter using fuzzy natural language commands to precisely select candidate results.

[0106] The system extracts the plot data (plot development ST1 to ST2) corresponding to the supporting character B between plot nodes V1 and V2, and outputs candidate content elements (STZ1, STZ2, STZ3, sorted by their fit with the original work) for the supporting character B within this interval based on the content generation model; the user selects the target content element (such as STZ3, which supports selecting multiple plots), and the system matches the scene list (including direct and indirect scene data) to generate the candidate scene positions (AD1, AD2, AD3) corresponding to the plot; the user selects the target scene position (AD3) according to their needs.

[0107] Based on the target content elements and target scene locations, the system generates an extended content data outline: "Supporting character B executes plot STZ3 in scene AD3 between plot nodes V1 and V2". It also adds details by combining character characteristic data (personality, behavioral logic), generates storyboards (F1...Fn), and divides the storyboards into timelines (e.g., F1 corresponds to 01:00, F2 corresponds to 05:00).

[0108] If the output is a video medium, a preset character model (such as a GAN face-swapping / multi-view reconstruction 3D model) is called to realize the storyboard as a keyframe scene. The transition animation between keyframes is supplemented by an optical flow algorithm. If frame or transition adjustments are needed, the model is annotated and regenerated. Finally, the frame sequence is concatenated to export the target video. If the output is text / comics or other mediums, after the storyboard is generated, the corresponding generation process is switched, and the target medium content is output based on the extended content data.

[0109] To implement the above embodiments, this application also provides a video generation apparatus. Figure 6 This is a schematic diagram of the structure of a video generation device 400 provided in an embodiment of this application. Figure 6 As shown, the device includes: The acquisition unit 410 is used to acquire multi-dimensional video data corresponding to the original video. The multi-dimensional video data is used to represent the full amount of structured information related to the plot, characters, and scenes in the original video. Building unit 420 is used to build a content generation model adapted to the original video based on multi-dimensional video data; The generation unit 430 is used to generate extended content data that conforms to the original video logic based on the content generation model and the content generation request input by the user, so as to generate the target video based on the extended content data.

[0110] In some embodiments, the multi-dimensional video data includes video plot data. The acquisition unit 410 is configured to: divide the original video into at least one video segment to be processed using a preset segmentation algorithm; extract keyframes from the first video segment to be processed in the at least one video segment to be processed to obtain a first keyframe set of the first video segment to be processed; perform speech recognition on the video audio corresponding to the first keyframe set to obtain a first audio text; determine the plot node corresponding to the first keyframe set based on the first audio text and the first subtitle text of the first keyframe set; and determine the video plot data based on the plot node.

[0111] In some embodiments, the multi-dimensional video data includes video character data, which includes at least one of character appearance data, character personality data, and character behavior data. The acquisition unit 410 is configured to: divide the original video into at least one video segment to be processed using a preset segmentation algorithm; extract keyframes from the first video segment to be processed in the at least one video segment to be processed to obtain a first keyframe set of the first video segment to be processed; perform character detection on the keyframes in the first keyframe set to obtain a first character in the first keyframe set; extract the character appearance data of the first character using image recognition technology, which includes at least one of clothing data, hairstyle data, and facial feature data; extract the character text corresponding to the first character, and determine the character personality data of the first character based on the character text and sentiment analysis technology, where the character text includes at least one of dialogue text and subtitle text; and obtain the joint coordinate information of the target character using human pose estimation technology, and determine the character behavior data of the first character based on the joint coordinate information and the character text.

[0112] In some embodiments, the multi-dimensional video data includes video scene data, which includes at least one of direct scene data and indirect scene data. The acquisition unit 410 is configured to: divide the original video into at least one video segment to be processed using a preset segmentation algorithm; extract keyframes from the first video segment to be processed in the at least one video segment to be processed to obtain a first keyframe set of the first video segment to be processed; perform scene recognition on the keyframes in the first keyframe set to determine the direct scene data of the first keyframe set, which includes at least one of scene type and scene location information; and infer the indirect scene data of the first keyframe set based on the character text of the first character in the first keyframe set and the video plot data corresponding to the first keyframe set.

[0113] In some embodiments, the generation unit 430 is configured to: parse a content generation request, determine the content generation requirements and contextual constraints that adapt to the content generation requirements in the content generation request; generate at least one candidate content element that conforms to the original video logic based on the contextual constraints, the content generation requirements, and the content generation model; respond to the user's selection operation on at least one candidate content element, determine the target content element selected by the user; and generate extended content data based on the target content element and multi-dimensional video data.

[0114] In some embodiments, the generation unit 430 is configured to: generate supplementary content videos based on extended content data and preset character models; merge the supplementary content videos with the original videos to obtain target videos when the content generation requirement is to fill in the gaps in the original videos' plot; and determine the supplementary content videos as target videos when the content generation requirement is to generate videos independently.

[0115] The methods and apparatus provided in the embodiments of this application have been described above. To implement the functions of the methods provided in the embodiments of this application, the electronic device may include a hardware structure and software modules, and may implement the above functions in the form of a hardware structure, software modules, or a hardware structure plus software modules. One of the above functions may be executed in the form of a hardware structure, software modules, or a hardware structure plus software modules.

[0116] Figure 7 This is a block diagram illustrating an electronic device 700 for implementing the video generation method described above, according to an exemplary embodiment. For example, the electronic device 700 may be a mobile phone, computer, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0117] Reference Figure 7 The electronic device 500 may include a communication interface 501, capable of interacting with other devices; a processor 502, connected to the communication interface 501 to interact with other devices, used to execute the methods provided by one or more of the above-described technical solutions when running a computer program; and a memory 503, on which the computer program is stored. Specifically, the specific processing procedure of the processor 502 can refer to the video generation method described in the above embodiments of this application.

[0118] Of course, in practical applications, the various components in electronic device 500 are coupled together through bus system 504. It can be understood that bus system 504 is used to realize the connection and communication between these components. In addition to a data bus, bus system 504 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 7 The general designated all buses as Bus System 504.

[0119] The memory 503 in this embodiment is used to store various types of data to support the operation of the electronic device 500. Examples of such data include any computer program used to operate on the electronic device 500.

[0120] The methods disclosed in the embodiments of this application can be applied to processor 502, or implemented by processor 502. Processor 502 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 502 or by instructions in the form of software. The processor 502 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 502 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 503. Processor 502 reads the information in memory 503 and combines its hardware to complete the steps of the aforementioned method.

[0121] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0122] Embodiments of this application also propose a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the video generation method described in the above embodiments of this application.

[0123] Embodiments of this application also propose a computer program product, including a computer program that is executed by a processor using the video generation method described in the above embodiments of this application.

[0124] Embodiments of this application also propose a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, it causes the electronic device to perform the video generation method described in the above embodiments of this application.

[0125] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0126] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0127] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0128] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (control method), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0129] It should be understood that various parts of the embodiments of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0130] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0131] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc.

[0132] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A video generation method, characterized in that, include: Obtain multi-dimensional video data corresponding to the original video, wherein the multi-dimensional video data is used to represent the full amount of structured information related to the plot, characters and scenes in the original video; Based on the multi-dimensional video data, a content generation model adapted to the original video is constructed; Based on the content generation model and the user's input content generation request, extended content data that conforms to the original video logic is generated, and the target video is generated based on the extended content data.

2. The method according to claim 1, characterized in that, The multi-dimensional video data includes video plot data, and obtaining the multi-dimensional video data corresponding to the original video includes: The original video is divided into at least one video segment to be processed using a preset segmentation algorithm; Keyframe extraction is performed on the first video segment to be processed from the at least one video segment to be processed to obtain a first set of keyframes for the first video segment to be processed. Speech recognition is performed on the video audio corresponding to the first keyframe set to obtain the first audio text. Based on the first voice text and the first subtitle text of the first keyframe set, determine the plot node corresponding to the first keyframe set; Based on the aforementioned plot nodes, the video plot data is determined.

3. The method according to claim 1, characterized in that, The multi-dimensional video data includes video character data, which includes at least one of character appearance data, character personality data, and character behavior data. The process of obtaining multi-dimensional video data corresponding to the original video includes: The original video is divided into at least one video segment to be processed using a preset segmentation algorithm; Keyframe extraction is performed on the first video segment to be processed from the at least one video segment to be processed to obtain a first set of keyframes for the first video segment to be processed. Perform character detection on the keyframes in the first keyframe set to obtain the first character in the first keyframe set; The character appearance data of the first character is extracted using image recognition technology. The character appearance data includes at least one of clothing data, hairstyle data, and facial feature data. Extract the character text corresponding to the first character, and determine the character personality data of the first character based on the character text and sentiment analysis technology. The character text includes at least one of dialogue text and subtitle text. By using human pose estimation technology, the keypoint coordinate information of the target character is obtained, and based on the keypoint coordinate information and the character text, the character behavior data of the first character is determined.

4. The method according to claim 1, characterized in that, The multi-dimensional video data includes video scene data, which includes at least one of direct scene data and indirect scene data. The process of obtaining multi-dimensional video data corresponding to the original video includes: The original video is divided into at least one video segment to be processed using a preset segmentation algorithm; Keyframe extraction is performed on the first video segment to be processed from the at least one video segment to be processed to obtain a first set of keyframes for the first video segment to be processed. Scene recognition is performed on the keyframes in the first keyframe set to determine the direct scene data of the first keyframe set, wherein the direct scene data includes at least one of scene type and scene location information; Based on the character text of the first character in the first keyframe set and the video plot data corresponding to the first keyframe set, indirect scene data of the first keyframe set is inferred.

5. The method according to claim 1, characterized in that, The process of generating extended content data that conforms to the original video logic based on the content generation model and the user-input content generation request includes: Parse the content generation request to determine the content generation requirements and contextual constraints that adapt to the content generation requirements. Based on the context constraints, the content generation requirements, and the content generation model, at least one candidate content element that conforms to the original video logic is generated. In response to the user's selection operation on the at least one candidate content element, determine the target content element selected by the user; Based on the target content elements and the multi-dimensional video data, extended content data is generated.

6. The method according to claim 5, characterized in that, The step of generating the target video based on the extended content data includes: Based on the expanded content data and the preset role model, supplementary content videos are generated; When the content generation requirement is to fill in the gaps in the original video's plot, the supplementary content video is merged with the original video to obtain the target video; If the content generation requirement is to generate a video independently, the supplementary content video will be identified as the target video.

7. A video generation apparatus, characterized in that, The device includes: The acquisition unit is used to acquire multi-dimensional video data corresponding to the original video, wherein the multi-dimensional video data is used to represent the full amount of structured information related to the plot, characters and scenes in the original video; The construction unit is used to construct a content generation model adapted to the original video based on the multi-dimensional video data. The generation unit is used to generate extended content data that conforms to the original video logic based on the content generation model and the content generation request input by the user, so as to generate the target video according to the extended content data.

8. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor is used to run the computer program, it performs the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images