71 results about "Storyboard" patented technology

The invention discloses a video editing method and related equipment, and the method comprises the steps: receiving an editing intention of a user, carrying out the semantic analysis of an intention analysis model, and generating an editing copywriting, a style and a structured instruction; the edited copywriting is split according to the edited copywriting; performing cross-modal analysis on video materials by using a video understanding model trained based on an open-source multi-modal model, and accurately positioning sub-lens segments; secondly, editing the sub-lens segments through an editing model to generate a preliminary sheet, performing quantitative scoring through a video scoring model, and obtaining a scoring result according to a multi-dimensional index; and finally, the result is fed back to the editing model, and iterative adjustment is carried out until a final slice is produced. According to the method, the editing efficiency is improved, the manual operation time is shortened, the material adaptability is enhanced, the personalized requirement is met, the editing effect consistency is improved, the video content understanding is deepened, the editing effect is optimized, the technical threshold is reduced, the existing technical problems are solved comprehensively, and an efficient, intelligent and personalized editing scheme is provided.

The invention relates to the technical field of artificial intelligence, and discloses a video generation method and device, equipment and a storage medium. The method comprises the following steps: splitting a target script through a large language model to obtain a split script, and extracting feature information of each role from the target script; generating a corresponding role image graph based on the feature information through a generation tool; respectively inputting the role image graph and each corresponding split script into a third language model to obtain a split graph corresponding to each split script; matching a corresponding target audio for each split script; combining each split script with the corresponding split image and the target audio to generate a corresponding split video; and combining all the split videos to obtain a target video. By adopting the method, matched videos with correct logic and plot coherence can be automatically and efficiently generated according to story characters, the individual requirements of users are met, and a large amount of labor cost, money cost and time cost can be saved.

The embodiment of the invention provides a virtual human video generation method and device, a computer program product and electronic equipment. The method comprises the following steps: identifying corresponding text content according to original audio, and determining skeleton posture graphs of multiple groups of human body actions; processing the text content and the skeleton posture graph through a large language model, and outputting split mirror planning information corresponding to the text content; the sub-mirror planning information comprises an overall style, scene description, figure description and sub-mirror attribute information of each sub-mirror; generating a scene dynamic graph based on the overall style and the scene description, and generating a character static graph based on the overall style and the character description; and according to the character static graph bound to each sub-mirror and the sub-mirror attribute information, generating a character motion graph of each sub-mirror, and combining the scene motion graph and the character motion graphs of the plurality of sub-mirrors to generate a virtual human video in the same scene. According to the technical scheme of the embodiment of the invention, the efficiency of generating the virtual human video can be improved.

The invention discloses a batch video generation method, an electronic device, a storage medium and a product, and relates to the technical field of video editing, and the method comprises the steps: responding to a first operation of a user, configuring materials and parameters for a sub-mirror template of a video generation task, and obtaining each sub-mirror; in response to a second operation of the user, determining a synthesis sequence of each sub-mirror to obtain each sub-mirror chain; for any one of the sub-mirror chains, generating a video clip corresponding to each sub-mirror in the sub-mirror chain, and combining the video clips into a target video corresponding to the sub-mirror chain; and after traversing each sub-mirror chain, obtaining a target video set of the video generation task. The technical problem that the diversity of generated videos is poor is solved.

The embodiment of the invention provides a video processing method and device, electronic equipment, a storage medium and a program product, and relates to the fields of video processing, artificial intelligence, cloud technology and the like. The method comprises the following steps: acquiring a to-be-processed video, performing lens splitting processing to obtain a plurality of sub-lens segments, generating a segment abstract of each sub-lens segment through a first big language model based on multi-modal information and first instruction information of each sub-lens segment, and generating a second segment abstract of each sub-lens segment based on the segment abstract of each sub-lens segment and acquired second instruction information, and determining at least one target sub-lens segment from the sub-lens segments through a trained second large language model, fusing the target sub-lens segments, and generating a target video segment corresponding to the to-be-processed video. According to the scheme of the invention, semantic understanding and content extraction are carried out based on the large language model by combining the various modal information of each sub-lens segment in the video, the target sub-lens segment meeting the selection condition in content is more accurately positioned, and the video production efficiency is improved.

The invention discloses an advertisement generation method, device and equipment and a readable storage medium, and the method comprises the steps: obtaining a target large model agent and a voice signal inputted by a user, and the agent comprises an intention recognition level, a frame generation level, a decision level and an editing level; based on the intention recognition level, intention recognition is carried out on the voice signal, and an advertisement strategy of the user is determined; generating a narrative framework based on the advertisement strategy by using the framework generation hierarchy; by utilizing the decision-making hierarchy, based on the narrative framework, obtaining each matched constituent element, and constructing a plurality of sub-mirror scripts; and based on the editing hierarchy, synthesizing each sub-mirror script to generate an advertisement. Therefore, the dependence on manual design in the advertisement generation process is reduced, the labor cost and the time cost are reduced, and the resource utilization efficiency is improved.

The invention discloses a multi-modal large model-based multimedia file generation method, and relates to the technical field of education content generation. Task alignment and evidence readiness, question analysis and question solving trajectory diagram construction, compiling into a unified instruction of a split white board and a parastyle, controlled generation, co-optimization in generation, posterior judgment, traceable release and learning write-back are sequentially completed; according to the scheme, a problem solving track graph is used as a unique true source, a semantic structure which can do a problem is mapped into a shot, blackboard writing and oral playing, and a suitable age target and a cognitive load are synchronously constrained; in the generation stage, the fact and safety scores are calculated according to the shot, local regeneration is triggered, and rework of the whole section is reduced; source credentials are written before publishing, and learning conditions are written back in combination with watching and evaluation, so that a correct, right-age, traceable and self-evolutionary closed loop is formed. And unified numbering, continuous input processing and output are realized, fixed-point backspacing and sampling inspection are supported, uncertainty and cost are reduced, and convenience is brought to campus distribution and home check verification.

The present technology pertains to presenting a storyboard user interface that includes a visual media timeline and a representation of a first frame on the timeline along with a prompt to generate visual media. The input prompt is then sent to a visual media generative response engine, which generates output visual media in response to the input prompt.

The embodiment of the invention provides a video generation method and device, equipment, a computer readable storage medium and a product, and the method comprises the steps: obtaining material contents determined by a user in response to a video generation operation triggered by the user; in response to a preview operation triggered by a user, displaying a story text content used for generating the target video in the first display interface, the story text content being generated by the large language model based on the material content; in response to a split picture generation operation triggered by a user, displaying a preset number of split pictures generated based on the story text content and preset first prompt content in the first display interface; and in response to a video generation operation triggered by a user, generating a target video based on the preset number of split pictures. Therefore, the user only needs to select the material content, the high-quality target video meeting the actual requirements of the user can be generated, and the generation efficiency of the target video is effectively improved.

The present application discloses a method and apparatus for generating a dynamic video, an electronic device, and a storage medium. The method comprises: acquiring a textual script of a storyboard; performing semantic expansion on the textual script by means of a large language model to obtain a plurality of scene descriptions, and generating storyboard still images corresponding to the scene descriptions; using a fusion inpainting model to perform inpainting processing on a target storyboard still image selected from the plurality of storyboard still images; generating a mask image of a dynamic area on the target storyboard still image selected by a user; by means of an image-to-video model, using feature information of the target storyboard still image and the mask image to generate a dynamic video of the storyboard according to image generation prompt information; performing video post-processing on the dynamic video; generating audio generation prompt information on the basis of a theme and style of the dynamic video, and generating a current video-based audio according to the audio generation prompt information by means of an audio generation model; and adding the current video-based audio to the dynamic video of the storyboard to obtain a final dynamic video of the storyboard.

A computer implemented method for generating video content based on natural language input is disclosed. The method includes receiving a natural language instruction describing one or more desired characteristics of a video. A structured script file comprising at least one story beat is generated using a natural language processing engine. A storyboard comprising one or more storyboard frames is created based on the structured script file. One or more virtual components are generated based on the storyboard. An intermediate video sequence comprising a visual component and an auditory component is created using virtual components and the storyboard. The intermediate video sequence is then refined to produce a modified video sequence by applying one or more post-processing effects.

To provide a system for easily converting text of a blog into a cartoon form.SOLUTION: A system includes a reception part, a generation part, a generation part including a character generation part and a background generation part, and an output part. The reception part inputs a text of a blog. The generation unit analyzes the text input by the reception unit and generates a storyboard. The generation unit including the character generation unit and the background generation unit generates the character and the background based on the storyboard generated by the generation unit. The output unit outputs the comic-style blog content generated by the generation unit.SELECTED DRAWING: Figure 1

A collaborative content generation system uses machine learning to generate a script and content depicting the performance of the script. A director may use the system to generate the script and optionally, may involve one or more collaborators who perform portions of the script. The system may use machine learning to generate or modify a script, a storyboard to visualize the story, a narrator (e.g., the narrator's voice), characters, music, sound effects, etc. A director may assign portions of the script to certain collaborators and select which of their recordings are interleaved into the final collaborative interleaved content series. The collaborators may independently perform their portions and provide clips of their performances to the system, which may then interleave the clips to produce the finalized content.

The invention relates to a novel movilization split mirror generation method based on a large language model, which comprises the following steps of: identifying roles, scenes and key props appearing for the first time in a text, and extracting specific descriptions of the roles, the scenes and the key props; roles, scenes and key props are updated and supplemented; the recognized content, the extracted description and the updated and supplemented setting information are structurally stored in a setting library to form a dynamically-updated and globally-shared knowledge source, so that a dynamic setting library is constructed and maintained; the large language model receives the original text content of the current chapter and all setting information related to the chapter and retrieved from the dynamic setting library; the large language model follows the description in the dynamic setup library and converts the narrative text into a structured split script to generate a movie and television split. According to the method, the novel text can be automatically converted into the movie and television play split script, and the production efficiency of movie and television works is improved.

The invention provides a digital human video generation method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, in particular to the fields of large models, artificial intelligence, content creation and the like. According to the specific implementation scheme, science popularization demand information is determined, and cue words of a large model are generated according to the science popularization demand information; generating a mouth broadcast script and a video split script according to the cue word through the large model; according to the oral playing script, obtaining a target material required by science popularization and an oral playing audio of the digital human; and based on the video split script, the target material and the oral audio of the digital person, generating a science popularization video of the digital person.

The present invention discloses a full-link text-to-video generation method and system based on a multimodal large model, which belongs to the field of artificial intelligence content generation technology. Through the collaborative work of multiple intelligent agents, user input text is analyzed, a cross-modal memory library is constructed, and the unified video and audio of the generated storyboard is ensured based on the memory library content, thereby realizing the full-process automatic generation from text to video. The implementation of this method includes the following steps: obtaining user text input; text analysis, through collaborative agents, dynamically extracting, analyzing, generating, associating, and storing multimodal information of images, texts, and audios from the input text to construct a multimodal memory library; generating storyboards, generating storyboard videos and audios according to the memory library; audio and video synthesis, and forming the final video after synchronous alignment of audio and video. The present invention can achieve narrative coherence in long video generation, improve the feature consistency of storyboards, enhance the consistency of cross-modal emotions, reduce manual intervention, and improve the efficiency of video production.

This invention proposes a method, system, and computer device for generating text-based videos, relating to the technical field of video processing. The method includes: acquiring user input parameters; generating anchor point prompts and anchor point character diagrams based on the user input parameters; evaluating the anchor point character diagrams until they pass evaluation; extracting information from the anchor point prompts to obtain a storyboard; evaluating the storyboards until they pass evaluation; generating audio and storyboard video based on the storyboards; evaluating the storyboard video until it passes evaluation; post-processing the storyboard video to obtain a post-processed video; and combining the post-processed video with the audio to generate a target video. This invention effectively provides a closed-loop evaluation node for text-based video generation, improving the quality of generated text-based videos.

A system is provided.SOLUTION: A system, comprising: means for receiving a plurality of pieces of prompt information input by a user; means for analyzing the prompt information; means for selecting a generation algorithm based on a result of the analysis; means for generating a storyboard using the selected generation algorithm; means for returning the generated storyboard to the user; and means for displaying and editing the returned storyboard.SELECTED DRAWING: Figure 1

This image is used to display a shooting plan such as a storyboard when shooting with a camera.

The present invention relates to a system supporting so that an AI system may analyze a storyboard and shot information, automatically generate development difficulty and development period information for each asset, and register same in a data management system, and a terminal device may identify the registered result. When an animation production system generates and transmits shots and asset information on the basis of the storyboard, the AI system analyzes the shots and the asset information to derive an asset list, the difficulty, and period information for each shot, and the data management system automatically registers and manages the corresponding information, so that the overall production process can be reviewed by means of the terminal device.

In various examples, generative AI-based systems and applications may be used to automatically transform basic storylines into comprehensive storyboards, as well as to automatically generate animatics based on the storyboards. For instance, input data (e.g., text, audio, etc.) representing a storyline may be obtained and analyzed using one or more AI models. The AI model(s) may segment the storyline into a plurality of scenes and generate storyboard frames for one or more of the scenes. In some examples, the AI model(s) may generate or otherwise determine scene descriptions, cinematographic details, and / or visual representations (e.g., images) for each of the scene(s), and this information may be included or otherwise used to generate the storyboard frames corresponding to each of those scene(s). Additionally, in some examples, the AI model(s) may perform extrapolation to generate intermediate scenes / storyboard frames to create animatics.

This invention discloses an automated short drama creation and partial editing method based on a multi-agent collaborative mechanism, specifically relating to the field of short drama creation and editing technology. The method involves: acquiring natural language commands input by the user; generating a structured script containing plot tension parameters and node verification information through an agent; dynamically determining the character feature fusion weights based on the plot tension parameters and generating storyboards by combining preset character reference images to maintain character consistency; generating audio based on the structured script and extracting timestamp information from the vocal units; constraining video generation with audio duration and applying lip-sync enhancement processing to the video frame intervals corresponding to the timestamp information; and, upon receiving an editing command, triggering partial regeneration or reusing existing materials based on the comparison results between the target modification node and the corresponding node verification information in the structured script to synthesize the final video.

The present disclosure provides a comic generation method, device, electronic device and storage medium, which relate to the field of artificial intelligence, specifically to technical fields such as NLP, large models, LLM, and deep learning. The specific implementation scheme is as follows: obtaining and displaying multiple storyboards; wherein the storyboards are obtained by splitting the story information associated with the target comic to be generated; in response to the confirmation operation of the multiple storyboards, sending a target comic style adapted to the target comic to the server; wherein the target comic style is used by the server to combine the multiple storyboards, determine the object characteristics, and obtain at least one character image that matches the object characteristics; receiving and displaying the character image sent by the server, and sending a target image determined based on the character image to the server; wherein the target image is used by the server to combine the multiple storyboards and the target comic style to generate the target comic; receiving and displaying the target comic sent by the server.

Features described herein generally relate to content production. Particularly, the present disclosure relates to facilitating video generation. Using machine-learning models, a storyboard can be generated from an inspirational video, video attributes can be determined for the storyboard, editing scores and actions can be determined for candidate videos, candidate videos can be edited based on the editing scores and actions, and the edited candidate videos can be combined to generate a video.

The application discloses a video generation method and device based on a data portrait and a storage medium, and belongs to the technical field of data processing. The method comprises the following steps: aggregating multi-source data of a target object from a data source, constructing a data portrait of the target object, inputting the data portrait into a story generation model, and generating a storyboard, wherein the storyboard comprises at least two scenes arranged based on a time axis, a scene is associated with a scene description, a side speech script and / or an emotional label, at least two media generation engines are called in parallel according to the emotional label and the scene description of the scene in the storyboard, voice dubbing, background music, animation materials and / or stylized materials are generated, and the generated voice dubbing, background music, animation materials and stylized materials are combined into a personalized video of the target object according to the time axis of the storyboard. The application combines the generated various materials into the personalized video of the target object based on the storyboard, and improves the overall coordination between various video generation scenes.

Systems and methods for producing a forensic storyboard that includes visual details of a criminal case. A forensic storyboard may be created from data within a forensic database. The forensic storyboard may be provided to a user through an interactive database. A user may be able to revise the forensic storyboard through the interactive webpage. A forensic storyboard may be output to a standalone file. A forensic storyboard may be used by an investigator to communicate details of a forensic investigation to superiors, colleagues, partners, other agencies, prosecutors, or judges.

An initial seed input for generation of a storyboard is received. A current image generation input is set the same as the initial seed input. A first artificial intelligence model is executed to automatically generate a current frame image based on the current image generation input. The current frame image and its corresponding description are stored as a next frame in the storyboard. A second artificial intelligence model is executed to automatically generate a description of the current frame image. A third artificial intelligence model is executed to automatically generate a next frame input description for the storyboard based on the description of the current frame image. The current image generation input is set the same as the next frame input description. Then, execution of the first, second, and third artificial intelligence models is repeated until a final frame image and its corresponding description are generated and stored.

An initial seed input for generation of a storyboard is received. A current image generation input is set the same as the initial seed input. A first artificial intelligence model is executed to automatically generate a current frame image based on the current image generation input. The current frame image and its corresponding description are stored as a next frame in the storyboard. A second artificial intelligence model is executed to automatically generate a description of the current frame image. A third artificial intelligence model is executed to automatically generate a next frame input description for the storyboard based on the description of the current frame image. The current image generation input is set the same as the next frame input description. Then, execution of the first, second, and third artificial intelligence models is repeated until a final frame image and its corresponding description are generated and stored.

The embodiment of the invention provides an image processing method and device, equipment, a computer readable storage medium and a product. The method comprises the steps of obtaining a to-be-processed image; performing content identification operation on the to-be-processed image through a preset image identification algorithm to obtain text description information corresponding to the to-be-processed image; generating at least one associated image associated with the content of the to-be-processed image based on the text description information; and splicing the to-be-processed image and the at least one associated image to obtain a target image. Therefore, the image processing mode of the to-be-processed image can be enriched, in addition, due to the fact that the association relation exists between the at least one associated image and the to-be-processed image, after the to-be-processed image and the at least one associated image are spliced, the target image can present the effect similar to a movie story board, and the image quality of the target image is improved.

A collaborative content generation system uses machine learning to generate a script and content depicting the performance of the script. A director may use the system to generate the script and optionally, may involve one or more collaborators who perform portions of the script. The system may use machine learning to generate or modify a script, a storyboard to visualize the story, a narrator (e.g., the narrator's voice), characters, music, sound effects, etc. A director may assign portions of the script to certain collaborators and select which of their recordings are interleaved into the final collaborative interleaved content series. The collaborators may independently perform their portions and provide clips of their performances to the system, which may then interleave the clips to produce the finalized content.