98 results about "Storyboard" patented technology

The invention discloses a video editing method and related equipment, and the method comprises the steps: receiving an editing intention of a user, carrying out the semantic analysis of an intention analysis model, and generating an editing copywriting, a style and a structured instruction; the edited copywriting is split according to the edited copywriting; performing cross-modal analysis on video materials by using a video understanding model trained based on an open-source multi-modal model, and accurately positioning sub-lens segments; secondly, editing the sub-lens segments through an editing model to generate a preliminary sheet, performing quantitative scoring through a video scoring model, and obtaining a scoring result according to a multi-dimensional index; and finally, the result is fed back to the editing model, and iterative adjustment is carried out until a final slice is produced. According to the method, the editing efficiency is improved, the manual operation time is shortened, the material adaptability is enhanced, the personalized requirement is met, the editing effect consistency is improved, the video content understanding is deepened, the editing effect is optimized, the technical threshold is reduced, the existing technical problems are solved comprehensively, and an efficient, intelligent and personalized editing scheme is provided.

The invention relates to the technical field of artificial intelligence, and discloses a video generation method and device, equipment and a storage medium. The method comprises the following steps: splitting a target script through a large language model to obtain a split script, and extracting feature information of each role from the target script; generating a corresponding role image graph based on the feature information through a generation tool; respectively inputting the role image graph and each corresponding split script into a third language model to obtain a split graph corresponding to each split script; matching a corresponding target audio for each split script; combining each split script with the corresponding split image and the target audio to generate a corresponding split video; and combining all the split videos to obtain a target video. By adopting the method, matched videos with correct logic and plot coherence can be automatically and efficiently generated according to story characters, the individual requirements of users are met, and a large amount of labor cost, money cost and time cost can be saved.

A method for generating storyboards is described. An extraction prompt is provided to a first generative neural network model. The extraction prompt is a text-based prompt that instructs the first generative neural network model how to identify timestamps of segments having related content within transcripts according to dialog within the transcripts. A transcript of a meeting is provided as an input to the first generative neural network model. Segment timestamps for identified segments within the meeting are received from the first generative neural network model based on the extraction prompt and the transcript. Segment images for the identified segments are generated using a second generative neural network model, wherein each of the segment images represents segment content within a corresponding identified segment.

The embodiment of the invention provides a virtual human video generation method and device, a computer program product and electronic equipment. The method comprises the following steps: identifying corresponding text content according to original audio, and determining skeleton posture graphs of multiple groups of human body actions; processing the text content and the skeleton posture graph through a large language model, and outputting split mirror planning information corresponding to the text content; the sub-mirror planning information comprises an overall style, scene description, figure description and sub-mirror attribute information of each sub-mirror; generating a scene dynamic graph based on the overall style and the scene description, and generating a character static graph based on the overall style and the character description; and according to the character static graph bound to each sub-mirror and the sub-mirror attribute information, generating a character motion graph of each sub-mirror, and combining the scene motion graph and the character motion graphs of the plurality of sub-mirrors to generate a virtual human video in the same scene. According to the technical scheme of the embodiment of the invention, the efficiency of generating the virtual human video can be improved.

The invention discloses a batch video generation method, an electronic device, a storage medium and a product, and relates to the technical field of video editing, and the method comprises the steps: responding to a first operation of a user, configuring materials and parameters for a sub-mirror template of a video generation task, and obtaining each sub-mirror; in response to a second operation of the user, determining a synthesis sequence of each sub-mirror to obtain each sub-mirror chain; for any one of the sub-mirror chains, generating a video clip corresponding to each sub-mirror in the sub-mirror chain, and combining the video clips into a target video corresponding to the sub-mirror chain; and after traversing each sub-mirror chain, obtaining a target video set of the video generation task. The technical problem that the diversity of generated videos is poor is solved.

The embodiment of the invention provides a video processing method and device, electronic equipment, a storage medium and a program product, and relates to the fields of video processing, artificial intelligence, cloud technology and the like. The method comprises the following steps: acquiring a to-be-processed video, performing lens splitting processing to obtain a plurality of sub-lens segments, generating a segment abstract of each sub-lens segment through a first big language model based on multi-modal information and first instruction information of each sub-lens segment, and generating a second segment abstract of each sub-lens segment based on the segment abstract of each sub-lens segment and acquired second instruction information, and determining at least one target sub-lens segment from the sub-lens segments through a trained second large language model, fusing the target sub-lens segments, and generating a target video segment corresponding to the to-be-processed video. According to the scheme of the invention, semantic understanding and content extraction are carried out based on the large language model by combining the various modal information of each sub-lens segment in the video, the target sub-lens segment meeting the selection condition in content is more accurately positioned, and the video production efficiency is improved.

Disclosed is a method for generating a storyboard based on a script text, which is performed by a computing device. The method may include obtaining the script text. The method may include performing clustering for text parts included in script text based on location information. The method may include determining a prompt information for each cluster generated by clustering based on the location information. The method may include generating the storyboard for each cluster based on the determined prompt information.

The invention discloses an advertisement generation method, device and equipment and a readable storage medium, and the method comprises the steps: obtaining a target large model agent and a voice signal inputted by a user, and the agent comprises an intention recognition level, a frame generation level, a decision level and an editing level; based on the intention recognition level, intention recognition is carried out on the voice signal, and an advertisement strategy of the user is determined; generating a narrative framework based on the advertisement strategy by using the framework generation hierarchy; by utilizing the decision-making hierarchy, based on the narrative framework, obtaining each matched constituent element, and constructing a plurality of sub-mirror scripts; and based on the editing hierarchy, synthesizing each sub-mirror script to generate an advertisement. Therefore, the dependence on manual design in the advertisement generation process is reduced, the labor cost and the time cost are reduced, and the resource utilization efficiency is improved.

The invention discloses a multi-modal large model-based multimedia file generation method, and relates to the technical field of education content generation. Task alignment and evidence readiness, question analysis and question solving trajectory diagram construction, compiling into a unified instruction of a split white board and a parastyle, controlled generation, co-optimization in generation, posterior judgment, traceable release and learning write-back are sequentially completed; according to the scheme, a problem solving track graph is used as a unique true source, a semantic structure which can do a problem is mapped into a shot, blackboard writing and oral playing, and a suitable age target and a cognitive load are synchronously constrained; in the generation stage, the fact and safety scores are calculated according to the shot, local regeneration is triggered, and rework of the whole section is reduced; source credentials are written before publishing, and learning conditions are written back in combination with watching and evaluation, so that a correct, right-age, traceable and self-evolutionary closed loop is formed. And unified numbering, continuous input processing and output are realized, fixed-point backspacing and sampling inspection are supported, uncertainty and cost are reduced, and convenience is brought to campus distribution and home check verification.

The present technology pertains to presenting a storyboard user interface that includes a visual media timeline and a representation of a first frame on the timeline along with a prompt to generate visual media. The input prompt is then sent to a visual media generative response engine, which generates output visual media in response to the input prompt.

The embodiment of the invention provides a video generation method and device, equipment, a computer readable storage medium and a product, and the method comprises the steps: obtaining material contents determined by a user in response to a video generation operation triggered by the user; in response to a preview operation triggered by a user, displaying a story text content used for generating the target video in the first display interface, the story text content being generated by the large language model based on the material content; in response to a split picture generation operation triggered by a user, displaying a preset number of split pictures generated based on the story text content and preset first prompt content in the first display interface; and in response to a video generation operation triggered by a user, generating a target video based on the preset number of split pictures. Therefore, the user only needs to select the material content, the high-quality target video meeting the actual requirements of the user can be generated, and the generation efficiency of the target video is effectively improved.

The present application discloses a method and apparatus for generating a dynamic video, an electronic device, and a storage medium. The method comprises: acquiring a textual script of a storyboard; performing semantic expansion on the textual script by means of a large language model to obtain a plurality of scene descriptions, and generating storyboard still images corresponding to the scene descriptions; using a fusion inpainting model to perform inpainting processing on a target storyboard still image selected from the plurality of storyboard still images; generating a mask image of a dynamic area on the target storyboard still image selected by a user; by means of an image-to-video model, using feature information of the target storyboard still image and the mask image to generate a dynamic video of the storyboard according to image generation prompt information; performing video post-processing on the dynamic video; generating audio generation prompt information on the basis of a theme and style of the dynamic video, and generating a current video-based audio according to the audio generation prompt information by means of an audio generation model; and adding the current video-based audio to the dynamic video of the storyboard to obtain a final dynamic video of the storyboard.

A computer implemented method for generating video content based on natural language input is disclosed. The method includes receiving a natural language instruction describing one or more desired characteristics of a video. A structured script file comprising at least one story beat is generated using a natural language processing engine. A storyboard comprising one or more storyboard frames is created based on the structured script file. One or more virtual components are generated based on the storyboard. An intermediate video sequence comprising a visual component and an auditory component is created using virtual components and the storyboard. The intermediate video sequence is then refined to produce a modified video sequence by applying one or more post-processing effects.

To provide a system for easily converting text of a blog into a cartoon form.SOLUTION: A system includes a reception part, a generation part, a generation part including a character generation part and a background generation part, and an output part. The reception part inputs a text of a blog. The generation unit analyzes the text input by the reception unit and generates a storyboard. The generation unit including the character generation unit and the background generation unit generates the character and the background based on the storyboard generated by the generation unit. The output unit outputs the comic-style blog content generated by the generation unit.SELECTED DRAWING: Figure 1

A collaborative content generation system uses machine learning to generate a script and content depicting the performance of the script. A director may use the system to generate the script and optionally, may involve one or more collaborators who perform portions of the script. The system may use machine learning to generate or modify a script, a storyboard to visualize the story, a narrator (e.g., the narrator's voice), characters, music, sound effects, etc. A director may assign portions of the script to certain collaborators and select which of their recordings are interleaved into the final collaborative interleaved content series. The collaborators may independently perform their portions and provide clips of their performances to the system, which may then interleave the clips to produce the finalized content.

The invention discloses a method, a system and a terminal for generating a comic with a consistent story, and the method comprises the steps: determining a target comic prompt word, carrying out the comic story generation processing and the split processing, and obtaining a plurality of split prompt words and a plurality of role prompt words; obtaining a preset diffusion model, and inputting each split prompt word and the plurality of role prompt words into the preset diffusion model to obtain a cross attention graph; performing mask segmentation processing on the cross attention map to obtain a role segmentation mask; constructing an expanded attention graph according to the cross attention graph and the role segmentation mask, and performing mask assignment processing and attention weight distribution processing to obtain an isolated self-attention graph; and performing cross attention feature extraction processing and weighted fusion processing on the isolated self-attention graph to obtain fusion features, and performing iterative diffusion denoising processing on the fusion features to obtain a target cartoon image with a consistent story. The visual quality of automatic cartoon generation can be effectively improved.

The invention relates to a novel movilization split mirror generation method based on a large language model, which comprises the following steps of: identifying roles, scenes and key props appearing for the first time in a text, and extracting specific descriptions of the roles, the scenes and the key props; roles, scenes and key props are updated and supplemented; the recognized content, the extracted description and the updated and supplemented setting information are structurally stored in a setting library to form a dynamically-updated and globally-shared knowledge source, so that a dynamic setting library is constructed and maintained; the large language model receives the original text content of the current chapter and all setting information related to the chapter and retrieved from the dynamic setting library; the large language model follows the description in the dynamic setup library and converts the narrative text into a structured split script to generate a movie and television split. According to the method, the novel text can be automatically converted into the movie and television play split script, and the production efficiency of movie and television works is improved.

The invention provides a digital human video generation method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, in particular to the fields of large models, artificial intelligence, content creation and the like. According to the specific implementation scheme, science popularization demand information is determined, and cue words of a large model are generated according to the science popularization demand information; generating a mouth broadcast script and a video split script according to the cue word through the large model; according to the oral playing script, obtaining a target material required by science popularization and an oral playing audio of the digital human; and based on the video split script, the target material and the oral audio of the digital person, generating a science popularization video of the digital person.

Provided are an electronic device and method for outputting vessel education content according to one embodiment of the present invention, wherein the electronic device may comprise a display, a sensor module for sensing a user input, and a processor configured to: when an execution request of an application for outputting vessel education content is received, control the display to display a basic screen on a 360-degree 3D image of a predetermined type of vessel; when the user input indicates a transparency setting of the 3D image, apply transparency of a value corresponding to the user input to the 3D image; control the display to display a plurality of preset viewpoints on the image to which the transparency is applied; as the user input indicates selection of a first viewpoint among the plurality of viewpoints, control the display to enlarge and display a portion of the 3D image corresponding to a first structure related to the first viewpoint; generate a first storyboard including a first description related to the first structure; and convert the first description into data in a designated form and output the data to the outside. Various other embodiments may be possible.

The present disclosure provides a comic image processing method and device, electronic device, and storage medium, and relates to the field of image processing technology. The comic image processing method includes: extracting the contour of the comic image to obtain the contour information of the comic image; performing polygon fitting based on the contour information, and determining the storyboard images based on the polygons obtained by fitting; sorting the storyboard images according to the position information of each storyboard image in the comic image; generating storyboard information based on the position information and sorting information of each storyboard image, and associating the storyboard information with the comic image. The present disclosure can improve the accuracy of storyboarding of comic images.

The present invention discloses a full-link text-to-video generation method and system based on a multimodal large model, which belongs to the field of artificial intelligence content generation technology. Through the collaborative work of multiple intelligent agents, user input text is analyzed, a cross-modal memory library is constructed, and the unified video and audio of the generated storyboard is ensured based on the memory library content, thereby realizing the full-process automatic generation from text to video. The implementation of this method includes the following steps: obtaining user text input; text analysis, through collaborative agents, dynamically extracting, analyzing, generating, associating, and storing multimodal information of images, texts, and audios from the input text to construct a multimodal memory library; generating storyboards, generating storyboard videos and audios according to the memory library; audio and video synthesis, and forming the final video after synchronous alignment of audio and video. The present invention can achieve narrative coherence in long video generation, improve the feature consistency of storyboards, enhance the consistency of cross-modal emotions, reduce manual intervention, and improve the efficiency of video production.

This invention proposes a method, system, and computer device for generating text-based videos, relating to the technical field of video processing. The method includes: acquiring user input parameters; generating anchor point prompts and anchor point character diagrams based on the user input parameters; evaluating the anchor point character diagrams until they pass evaluation; extracting information from the anchor point prompts to obtain a storyboard; evaluating the storyboards until they pass evaluation; generating audio and storyboard video based on the storyboards; evaluating the storyboard video until it passes evaluation; post-processing the storyboard video to obtain a post-processed video; and combining the post-processed video with the audio to generate a target video. This invention effectively provides a closed-loop evaluation node for text-based video generation, improving the quality of generated text-based videos.

A system and method for generating a digital cybersecurity artifact includes selectively executing an automated cybersecurity investigation workflow based on a probable cybersecurity threat type of a cybersecurity event, wherein an output of the automated cybersecurity investigation workflow includes one or more corpora of investigation findings data in response to executing the automated cybersecurity investigation workflow; selectively instantiating a digital cybersecurity artifact of a plurality of digital cybersecurity artifacts based on the probable cybersecurity threat type of the cybersecurity event, wherein the digital cybersecurity artifact includes a plurality of distinct regions electronically mapped to one or more threat type-specific content automations that, when executed, install investigation findings data into the plurality of distinct regions of the plurality of distinct regions of the digital cybersecurity artifact with selective subsets of investigation findings data of the one or more corpora of investigation findings data.

A system is provided.SOLUTION: A system, comprising: means for receiving a plurality of pieces of prompt information input by a user; means for analyzing the prompt information; means for selecting a generation algorithm based on a result of the analysis; means for generating a storyboard using the selected generation algorithm; means for returning the generated storyboard to the user; and means for displaying and editing the returned storyboard.SELECTED DRAWING: Figure 1

Systems and methods, and non-transitory computer-readable medium storing instructions that, when executed, causes a processor to perform operations, including displaying a storyboard via a user output device. The story board may include a plurality of frames. The frames may include an illustration. The illustration may include a first state and a second state. The operations may further include changing the illustration within the frame from the first state to the second state based on a scroll command from a user input device from a first frame to a second frame of the plurality of frames.

This image is used to display a shooting plan such as a storyboard when shooting with a camera.

To provide a computer program, an information processing device and an information processing method which facilitate creation of a storyboard.SOLUTION: A computer program displays, on a touch-panel integrated display of a computer, multiple objects corresponding to respective scenes forming a storyboard chronologically together with thumb nails of images rendered for respective scenes, receives on the display an object that corresponds to a scene to be edited from among the objects, displays an editing screen for the scene corresponding to the selected object on the same screen as the multiple objects, receives rendering editing to the scene on the displayed editing screen, displays in the same screen a reproduction button for reproducing the multiple scenes as a moving image according to a set time axis, and when the reproduction button is selected on the display, reproduces the multiple scenes as a moving image.SELECTED DRAWING: Figure 5

The present invention relates to a system supporting so that an AI system may analyze a storyboard and shot information, automatically generate development difficulty and development period information for each asset, and register same in a data management system, and a terminal device may identify the registered result. When an animation production system generates and transmits shots and asset information on the basis of the storyboard, the AI system analyzes the shots and the asset information to derive an asset list, the difficulty, and period information for each shot, and the data management system automatically registers and manages the corresponding information, so that the overall production process can be reviewed by means of the terminal device.

In various examples, generative AI-based systems and applications may be used to automatically transform basic storylines into comprehensive storyboards, as well as to automatically generate animatics based on the storyboards. For instance, input data (e.g., text, audio, etc.) representing a storyline may be obtained and analyzed using one or more AI models. The AI model(s) may segment the storyline into a plurality of scenes and generate storyboard frames for one or more of the scenes. In some examples, the AI model(s) may generate or otherwise determine scene descriptions, cinematographic details, and / or visual representations (e.g., images) for each of the scene(s), and this information may be included or otherwise used to generate the storyboard frames corresponding to each of those scene(s). Additionally, in some examples, the AI model(s) may perform extrapolation to generate intermediate scenes / storyboard frames to create animatics.

A method for generating storyboards is described. An extraction prompt is provided to a first generative neural network model. The extraction prompt is a text-based prompt that instructs the first generative neural network model how to identify timestamps of segments having related content within transcripts according to dialog within the transcripts. A transcript of a meeting is provided as an input to the first generative neural network model. Segment timestamps for identified segments within the meeting are received from the first generative neural network model based on the extraction prompt and the transcript. Segment images for the identified segments are generated using a second generative neural network model, wherein each of the segment images represents segment content within a corresponding identified segment.