340 results about "Video production" patented technology

The invention discloses a short video production method and system based on artificial intelligence, and relates to the technical field of multimedia processing, and the method comprises the steps: inputting a split script into a multi-modal generation engine, employing a lower triangle space-time attention matrix to generate a visual stream, and generating a text stream and an audio stream; calculating a neural conduction delay amount between the visual stream and the audio stream based on an audio-visual perception delay prediction model, and generating a corrected audio stream through time axis forward compensation; calculating a cross-modal attention weight matrix according to the node association strength of the dynamic causal graph, and projecting the text stream and the corrected audio stream to a joint feature space to generate a multi-modal fusion feature; and inputting the multi-modal fusion features into a format adaptation engine, and after time-space consistency verification, embedding a neural implicit watermark and encoding the neural implicit watermark into a short video. According to the invention, the semantic consistency and coordination among the text, the audio and the visual content are improved, so that high-quality and high-intelligence automatic generation of the short video is realized on the whole.

The invention provides a video production method and device for automatically explaining static PPT by a digital human, computer equipment and a computer storage medium, and relates to the technical field of video processing. Comprising the following steps: extracting first target text information in each sub-image from a sequence image set of static PPT; converting the first target text information in each subimage into voice, and synthesizing the obtained initial audio files corresponding to each subimage to obtain a target audio file for the static PPT; wherein a preset pause duration is added between the initial audio files in the target audio file, and a corresponding relation between the audio time of each initial audio file and the display time of each sub-image is constructed; adding a PPT transition animation with a preset pause duration between the sub-images to obtain an initial PPT video for the static PPT; and based on the target audio file, controlling the virtual digital human to explain first target text information in the initial PPT video to obtain a target PPT video. According to the method, the video playing fluency can be improved.

The invention provides a video generation method and device based on multi-agent cooperation and agents, relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning, large models, agents, AIGC, element universe and the like, and is applied to application scenes such as wisdom education, video production, animation production and the like. The method comprises the following steps: executing a target sub-task in a target task by utilizing at least one target agent to obtain a target sub-execution result; and generating a target video based on the target sub-execution result, the target agent executing the following operations: obtaining sub-task demand information for the target sub-task; determining a subtask execution element for controlling the execution process of the target subtask by using a first subagent of the target agent based on the subtask demand information; and utilizing a second sub-agent of the target agent to execute the target sub-task based on the sub-task execution element to obtain a target sub-execution result.

The invention discloses a text-to-video full-link generation method and system based on a multi-modal large model, and belongs to the technical field of artificial intelligence content generation. Through cooperative work of multiple agents, analysis of a text input by a user and construction of a cross-modal memory library, unification of videos and audios for generating a sub-mirror is ensured based on memory library content, and the generation efficiency of the sub-mirror is improved. The full-process automatic generation from the text to the video is realized; the method is implemented by the following steps of: acquiring text input of a user; text analysis: dynamically extracting, analyzing, generating, associating and storing image-text-tone multi-modal information from an input text through an Agent working cooperatively, and constructing a multi-modal memory library; generating a split mirror, and generating a split mirror video and an audio according to the memory bank; and performing audio and video synthesis and audio and picture synchronous alignment to form a final video. According to the method, the narrative continuity of long video generation can be realized, the feature consistency of the split image is improved, the consistency of cross-modal emotion is enhanced, manual intervention is reduced, and the video production efficiency is improved.

A full-automatic movie and television video production system based on digital actors belongs to the field of artificial intelligence, computer graphics and movie and television production and comprises a creative engine unit, a production unit, an intelligent shooting unit, a post-processing unit and a quality inspection and rendering unit. The method has the advantages that 1, the manufacturing efficiency is greatly improved, the traditional film and television manufacturing period is usually 6-12 months, the manufacturing period can be shortened to 4 hours by the method, and the efficiency is improved by about 1000 times; 2, the manufacturing cost is obviously reduced: the manufacturing cost of traditional movies and other movies is usually 5 million to 20 million dollars, but the method only needs the computing power cost, and the cost is less than one percent of that of the traditional movies; 3, the style consistency is guaranteed, traditional film and television production depends on personal ability, the style consistency is difficult to guarantee, and the style consistency is strictly guaranteed through an algorithm and is improved by 100%.

The invention provides an interaction method and device based on agent cooperation, an agent and a storage medium, belongs to the technical field of artificial intelligence, particularly relates to the technical fields of deep learning, large models, agents, AIGC and the like, and is applied to application scenes such as wisdom education, video production, wisdom medical treatment and the like. The intelligent agent cooperation-based interaction method comprises the following steps: performing intention understanding on initial input information for a target object to obtain an initial intention understanding result; performing interaction mode demand detection on the initial intention understanding result by using a first agent to obtain an interaction mode demand attribute; an interaction element matched with the interaction mode demand attribute is called to interact with the target object, interaction information representing the demand intention of the target object is obtained, and the interaction element is used for responding to the interaction behavior; and controlling the second agent to execute the target task based on the interaction information to obtain an execution result, and pushing the execution result to the target object.

The invention discloses a method, a system and equipment for generating an oral video of a digital human, and a medium, and the method comprises the steps: obtaining oral copywriting and video material data, and analyzing and determining a timestamp of the copywriting in a video material through a multi-mode large model; converting the copywriting into audio data, preprocessing the audio data, and combining the audio data with the timestamp to generate first video data; and generating a digital human according to a user demand, and combining the digital human with the first video after image matting processing to obtain an oral playing video. According to the method, a traditional template generation mode is broken through, and customized production of the digital human broadcast video is realized through a multi-modal semantic matching and personalized digital human generation technology; and meanwhile, audio and video accurate synchronization, high-quality image matting and synthesis technologies are adopted, so that the content adaptability and the video quality are guaranteed, and the flexibility, the efficiency and the effect of digital population broadcast video production are remarkably improved.

The embodiment of the invention provides a video editing method and device, a storage medium and a program product. In the embodiment of the invention, a first video template used for describing a rendering rule and a hierarchical relationship of a group of video materials is downloaded to local terminal equipment, and video editing software is imported; opening the first video template in an editing area of the video editing software, and displaying a first video generated based on the first video template in a preview area of the video editing software; further, determining a modified rendering rule in any target materialized template and / or a modified position relationship between any two target materialized templates by responding to a modification operation on the first video template, and editing the first video according to the modified rendering rule and / or the modified position relationship, according to the video editing method and device, the video production efficiency and diversity can be improved, the dependence on professional equipment and manual intervention is reduced, and the diversified video requirements of high efficiency and low cost are met.

The invention discloses a short video automatic generation method and device based on a new media content creation model, and a server side method comprises the steps: receiving a short video generation request sent by a client side, wherein the short video generation request carries a video production description text and a user identifier; inputting the video production description text into a pre-fine-tuned large language model, and outputting video production scheme information; searching temporary video materials and system materials uploaded by the user from a BI system through the user identifier; under the condition that the temporary video material is not empty, inputting the video production scheme information, the temporary video material and the system material into a video generation model; or under the condition that the temporary video material is empty, inputting the video production scheme information and the system material into the video generation model; and outputting the target short video, and sending the target short video to the client. Therefore, by adopting the embodiment of the invention, the generated short video can show rich layering sense and emotion tension, so that the quality of the short video is improved.

The invention provides a video generation method and system of a self-adaptive model and related equipment. The method comprises the following steps: constructing a multi-model aggregation video generation platform; performing model characteristic labeling of a preset table on each model in the multi-model aggregation video generation platform; receiving contents and demands input by a user; based on the content and the demand input by the user and in combination with the model characteristic label, performing model matching of adaptive modes including a user preference priority mode, a resource priority mode, a quality priority mode, a cost priority mode and a fault-tolerant priority mode, and obtaining a recommended video generation model; and processing the content input by the user based on the recommended video generation model to obtain a target video. According to the technical scheme, the user only needs to input the content and the demand, the multi-model aggregation video generation platform can adaptively recommend the appropriate model based on the content and the demand input by the user, and the efficiency and the convenience of video production of the user are improved.

The invention provides a video processing method and device, equipment and a storage medium, and the method comprises the steps: segmenting a video material into a plurality of semantically coherent video clips, automatically extracting related video clips from the video material according to the text content provided by a user, and generating corresponding subtitles and voice broadcast ports matched with the related video clips, and finally synthesizing a section of finished video with clear logic and coherent content. According to the scheme, full-process automation from video material generation to final video file generation is realized, a large number of video materials can be quickly processed, the tedious manual editing process in traditional video production is reduced, the dependence on manual intervention is greatly reduced, the operation cost is reduced, the video production process is simplified, the video production efficiency is improved, and the user experience is improved. And the quality and the efficiency of finished products are also improved.

The invention provides a projection transformation method for hexahedral panoramic data to spherical panoramic data, and the method achieves the seamless splicing of panoramic images of six surfaces of a cube and the mapping of the panoramic images to a spherical coordinate system through the determination of a direction vector, the normalization of the vector, the conversion of spherical coordinates, the projection to an equidistant histogram, and an illumination and color correction algorithm. The problems of splicing gaps, deformation, texture distortion and the like in a traditional method are solved. The method is not only widely applied to the fields of virtual reality, augmented reality, panoramic video production and the like, but also greatly expands the application scenes, including but not limited to panoramic roaming, streetscape roaming experience, integrated use of specific roaming plug-ins, seamless embedding on a map platform and the like, and has wide application prospects. Therefore, the display effect of the panoramic data and the immersive experience of the user are remarkably improved.

The invention provides an automatic video generation method and device, equipment and a medium, and relates to the technical field of video production. The method comprises the following steps: acquiring video related information input by a user, and generating copywriting content according to the video related information and a preset copywriting script material library; according to a preset sound material library, converting the copywriting content into audio by using a text-to-voice conversion technology to obtain an audio material; according to the audio material, a digital human-driven technology is utilized, and a preset image material library and a video sub-mirror material library are combined, so that a visual material synchronized with the copywriting content is obtained; according to a preset packaging element material library, obtaining a packaging material matched with the copywriting content; and performing video synthesis and packaging on the audio material, the visual material and the packaging material to obtain a final video. According to the method, related materials can be generated through accurate matching and intelligent recommendation according to user requirements and recent popularity, a mixed cutting type and generation type material fusion arrangement mechanism is combined, one-key film formation is achieved, and the video production efficiency is improved.

The invention discloses a video processing method and device, a storage medium and electronic equipment, and relates to the technical field of image processing, in a processing mode corresponding to a processing demand, a diffusion model generation graph is used as a reference graph, and super-resolution processing is performed on a single-scene video clip and a reference image through a video super-resolution model, so that the video processing efficiency is improved. Due to the fact that super-resolution processing is an image processing mode that detail information in the reference image is extracted and supplemented to super-resolution, the detail richness of the image is improved, time consumption and instability caused by frame-by-frame generation of a diffusion model are effectively avoided, video production is carried out according to the reconstructed resolution video sequence, and the video quality is improved. The purpose of improving the resolution of video production is achieved.

The invention discloses a video generation method and device based on AIGC and a storage medium, and the method comprises the steps: extracting information expressed in lyrics of a song, generating a story summary according to the information expressed by the lyrics, and generating a corresponding story plot according to each lyric segment; inputting the lyric segments and the corresponding story lines into the artificial intelligence model, generating sub-shots corresponding to the lyric segments, and generating corresponding images / videos according to the sub-shots; and sorting and processing the images / videos corresponding to the sub-shots according to a time sequence to obtain song videos corresponding to the songs. According to the method, the video conforming to the artistic conception of the song can be efficiently generated, and the song video production efficiency and quality are improved.

The invention relates to a video production method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: obtaining a session between a user and a virtual object, and generating a corresponding initial video material based on the session; in response to an editing operation of the user on the initial video material, obtaining an edited video material; and generating a target video file based on the edited video material. The method can improve the flexibility of video production.

The invention relates to the technical field of videos, and discloses a video generation method and system based on an AI script, and the method comprises the steps: carrying out the frame-by-frame analysis of an input video through a convolutional neural network, extracting a visual feature vector, building the inter-frame time sequence correlation through a recurrent neural network, achieving the segmentation of video segments, and carrying out the semantic annotation, and constructing a multi-dimensional feature vector space. In combination with personalized requirements input by a user, the matching degree is calculated by applying a cosine similarity algorithm, and candidate segments are screened and clustered and grouped. And finally, determining an optimal fragment combination sequence, and generating a personalized video meeting user requirements. According to the method, the video content can be intelligently analyzed, the intention of the user is accurately grasped, the personalized video is automatically generated, and the video production efficiency and quality are improved.

The embodiment of the invention provides a video processing method and device, electronic equipment, a storage medium and a program product, and relates to the fields of video processing, artificial intelligence, cloud technology and the like. The method comprises the following steps: acquiring a to-be-processed video, performing lens splitting processing to obtain a plurality of sub-lens segments, generating a segment abstract of each sub-lens segment through a first big language model based on multi-modal information and first instruction information of each sub-lens segment, and generating a second segment abstract of each sub-lens segment based on the segment abstract of each sub-lens segment and acquired second instruction information, and determining at least one target sub-lens segment from the sub-lens segments through a trained second large language model, fusing the target sub-lens segments, and generating a target video segment corresponding to the to-be-processed video. According to the scheme of the invention, semantic understanding and content extraction are carried out based on the large language model by combining the various modal information of each sub-lens segment in the video, the target sub-lens segment meeting the selection condition in content is more accurately positioned, and the video production efficiency is improved.

The invention relates to a short video generation method and system and a terminal, and relates to the technical field of video production, and the method comprises the steps: obtaining image material information and short video demand information uploaded by a common user; calling demand style information and a demand duration value based on the short video demand information; determining material demand characteristics according to the demand style information; performing identification according to the material demand features and the image material information to determine selected material information; calling style demand material information from a preset video material library based on the demand style information, and combining the selected material information with the style demand material information to form use material information; and synthesizing and editing the use material information according to the demand style information and the demand duration value to generate short video output information, and outputting the short video output information to a terminal held by a common user for display. The method and the device have the effect of facilitating a common user to make the short video.

The invention discloses an online teaching video production method and device based on a virtual scene, and a medium, belongs to the technical field of teaching video production, and is used for solving the technical problem of high curriculum abandoning rate caused by low construction efficiency, low content updating rate and low knowledge transmission accuracy of an existing online curriculum. The method comprises the steps of customizing a digital teacher image of a target teacher based on modeling software; wherein the digital teacher image comprises a 3D digital teacher image and a 2D digital teacher image; constructing a virtual teaching scene, and training a virtual scene model to identify a user instruction so as to adjust features in the virtual teaching scene; based on the made teaching courseware, generating a corresponding digital human control script and a voice explanation script; placing the digital teacher image and the virtual teaching scene into a virtual environment, and fusing the digital teacher image and the virtual teaching scene with the digital human control script and the voice explanation script to form a virtual teaching model; and performing video rendering on the virtual teaching model to obtain a teaching video.

The invention relates to the technical field of digital people, and provides a method and equipment for generating a digital people video, which can be applied to a digital marketing scene. According to the method, the generation of the digital human video is divided into an audio generation process and a video generation process, when the audio is generated, a target text, an emotion parameter and a reference audio are taken as input, and the target audio of the target text is broadcasted by the sound of a reference person in the reference video and a set emotion, so that sound cloning is realized, the degree of violation of the generated digital human video is reduced, and the user experience is improved. And the text features are extracted from two dimensions of characters and pronunciation, so that the pronunciation accuracy during sound cloning is ensured. When the video is generated, the cloned target audio and the cloned reference video are used as input, and the fidelity of the digital human video is improved through expression and mouth shape matching, so that a reference person can quickly and efficiently generate vivid digital human videos with different broadcast contents only by shooting one reference video without shooting one by one, thereby reducing the manufacturing cost of the digital human video, and improving the user experience. And the video production efficiency is improved.

The present disclosure provides a video generation method and apparatus, a device, and a medium. The video generation method comprises: extracting, from a video to be processed, a target video clip matched with beat information of an original video; and acquiring a target video on the basis of the target video clip and the material features of the original video, wherein the material features at least comprise audio features of the original video. The method can guarantee that the audio and the beat of the obtained target video are matched, and the purpose of directly applying the original video is achieved, improving the target video production efficiency.

The invention discloses a full-automatic video generation system and method based on digital twinning and AI technologies, relates to the technical field of image data processing, and aims to solve the problem that full-process automatic production is difficult to realize in existing meteorological and ocean forecast video production. The method comprises the following steps: automatically acquiring structured data from an external digital twin API (Application Program Interface) through an automatic data acquisition module; a broadcast text is automatically generated and synthesized into an audio file based on the structured data through an AI audio automatic generation module; through a dynamic visualization rendering and capturing module, generating a dynamic visualization picture based on structured data driving, capturing the picture to form a continuous image frame sequence, and recording a timestamp of specific element switching in the picture at the same time; and synchronously aligning the voice audio file and the continuous image frame sequence according to the timestamp through an audio and video intelligent packaging module, mixing background music, and packaging and outputting a final video file. And full-process automatic production from structured data to flaking is realized.

The invention provides a video generation method, a training method and an intelligent agent based on a hybrid expert architecture, and relates to the technical field of artificial intelligence, in particular to the technical fields of large models, computer vision, video production, meta universe and the like. The video generation method based on the hybrid expert architecture comprises the following steps: receiving demand information related to a target object; processing the demand information by using a large model determined based on the hybrid expert architecture, generating a target video matched with the demand information, the large model comprising a routing layer and a plurality of expert layers, and determining a target expert layer in the plurality of expert layers by using the routing layer to perform semantic understanding on the demand information, and executing a video generation task by using the target expert layer according to the demand information to obtain a target video.

The invention relates to the technical field of vehicles, in particular to an automobile Vlog video generation method and device, a vehicle and a computer program product, and the method comprises the steps: obtaining real-time image data collected by a vehicle-mounted sensor, and recognizing a key event and a key scene in the real-time image data; generating a logic framework of video editing according to the identified key event and key scene, and generating an editing instruction of the Vlog video through a preset intelligent video editing algorithm based on the logic framework; and automatically editing and rendering the key event and the key scene according to the editing instruction to generate a target Vlog video. Therefore, the problems of low video production efficiency and low automation degree of a vehicle-mounted video generation scheme in related technologies are solved, the video generation efficiency is improved, the video content is enriched, the continuity and narrative property of the video content are ensured, and the user experience is improved.

The invention discloses a video processing method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the photographing of a virtual scene through responding to a scene photographing operation for the virtual scene, and obtaining a basic video; in response to a selection operation for a first target virtual object in the at least one virtual object, displaying a first object recording page corresponding to the first target virtual object, the first object recording page comprising a virtual scene corresponding to the basic video and the first target virtual object located in the virtual scene; in response to a control operation for the first target virtual object, controlling the first target virtual object to execute a first target action, and obtaining first action record information of the first target virtual object; and updating the basic video based on the basic video and the first action record information of the first target virtual object to obtain a target video. The video production steps are simplified, a large amount of manpower and time are saved, and the video production efficiency is improved.

A system (100) for providing a recommended video production (80) is disclosed herein. The system (100) comprises an overview camera (10) for capturing an overview recording, additional camera(s) (15a-n) for capturing additional recordings, and a back-end video production unit (44). The back-end video production unit is configured to receive the recordings being synchronized in time, apply image processing algorithm(s) to the overview recording for determining interest levels, wherein interest levels are associated with real-world events occurring within and around subareas of the real-world target field (20), select video contents among the recordings, wherein the selection is based on the interest levels as defined by the processed overview video recording, and based on said selection, produce the recommended video production (80) that, for each time unit, comprises video contents reflecting a currently highest interest level among the overview video recording and the additional video recording(s).

The embodiment of the invention provides a video generation method, system and device, and the method comprises the steps: generating a video subtitle file and an initial video according to an obtained video script; according to the video subtitle file, determining at least one animation material template from an animation material library, determining target attribute information of each animation material template, and generating at least one target animation material through each target attribute information; generating a target video corresponding to the video script according to each target animation material, the video subtitle file and the initial video; a manual dynamic effect editing process after video production is advanced to an automatic node of video generation and production, so that the dynamic effect generation efficiency of the whole video production is greatly improved, and the dynamic effect production efficiency of the video is greatly improved.

Embodiments of the present disclosure disclose a video generation method and an electronic device. The method comprises: creating a 3D virtual set environment on the basis of a 3D development engine; after receiving a video generation requirement text, generating a script by means of mutual coordination between multiple intelligent agents on the basis of an artificial intelligence (AI) large model, the multiple intelligent agents being used for simulating multiple functional jobs involved in an artificial video production process, the script comprising a character introduction in the script, actor lines, and a character position, action and shot types corresponding to the actor lines; performing structural processing on the script, and simulating a video shooting process in the 3D virtual set environment on the basis of the structured script, so as to generate a target video. By means of the embodiments of the present disclosure, video generation quality and character consistency can be improved.

There is provided a learning method, an information processing device, and a recording medium that can achieve high-speed learning by a neural network. The information processing device renders a plurality of depth images based on a plurality of different viewpoints from low-precision three-dimensional data, and performs learning processing of a neural network that generates high-precision three-dimensional data from a two-dimensional image, based on the plurality of depth images. The present disclosure can be applied, for example, to a technology for creating 3D assets for large-scale outdoor video production.