Automatic video editing method based on multi-agent cooperation and related device

Abstract

The invention belongs to the technical field of artificial intelligence and multimedia processing, and discloses an automatic video editing method based on multi-agent collaboration and a related device. The automatic video editing method comprises the following steps: receiving a video editing demand input by a user in a natural language form through a master control agent; performing summary analysis on the media material to be processed, and receiving a structured analysis result returned by the multi-modal analysis agent; carrying out editing planning, generating an ordered execution sequence formed by calling a series of atomization editing tools, calling an atomization editing tool set, and constructing a complete editing project file; and rendering the edited item file through a rendering engine, and outputting a target video file. According to the technical scheme disclosed by the invention, the technical problems existing in the aspects of long video processing, personalized demand response, end-to-end automatic execution capability and the like in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence and multimedia processing, and particularly relates to an automatic video editing method based on multi-agent cooperation and a related device. BACKGROUND

[0002] With the explosive growth of digital media content, video editing has become a core link in the content creation process. The existing traditional video editing highly depends on manual operation, which not only consumes time and effort, but also has high requirements for the professional skills and creative experience of the operator, limiting the production efficiency of high-quality video content. To address the above challenges, the industry has carried out a series of explorations of automatic editing technology. At present, the existing related technologies can be mainly divided into two categories: The first category is a functional point auxiliary tool. This kind of tool applies AI technology to a specific and independent link in the editing process. For example, in the aspect of intelligent material retrieval, subtitles are automatically generated through voice recognition, or materials are classified and labeled through image recognition technology to facilitate manual searching. In the aspect of segment screening assistance, AI is used to identify and exclude silent, repetitive or shaking segments in the video. In the aspect of effect generation, AI-driven filters, one-key color grading or special effect generation tools. These tools improve the efficiency of a single link to some extent, but they are essentially "tool box" type auxiliary functions and do not change the core mode that the editing workflow still needs manual guidance and connection, and cannot realize end-to-end full-process automation.

[0003] The second category is template-based video generation. This kind of application allows users to upload a small amount of materials and select a preset template (exemplarily, such as "birthday review", "travel Vlog", etc.), and the system automatically fits the materials into the template and adds music to quickly generate a short video. Although this kind of application realizes a certain degree of "automation", its limitations are also very obvious, mainly including: (1) strictly limited by the structure and style of the template, unable to respond to the personalized and complex creative intentions of the user, with the defect of low creative freedom; (2) usually only performs simple duration matching or random splicing on the materials, lacks deep understanding and organic combination of video pictures and audio content, and has the defect of shallow content understanding; (3) usually only suitable for short video production of specific topics, unable to handle complex narratives of long videos or non-standard creative needs, with the defect of limited scene and process; (4) the user still needs to manually select the template, upload the materials, etc., and cannot complete all the work just by a piece of text instruction, with the defect of imperfect automation degree.

[0004] Furthermore, while existing AI-generated content (AIGC) technologies can create video content from scratch, their architecture and methodology are not designed for processing and editing existing large-scale media assets (such as feature films and conference recordings), i.e., they are designed for video editing. Therefore, in specific scenarios of automated video editing, the following key technical challenges remain unresolved: Context window bottleneck: As the core of planning, the large language model has a limited context window, which makes it unable to directly load and understand long video files containing millions of frames of information, causing it to "go blind" when processing long videos. The challenge of high-order semantic localization: How to efficiently and accurately locate segments that meet the user's abstract artistic needs (such as "all the moments when the protagonist's emotions break down" or "shots full of suspense") in massive amounts of unstructured video data is a major challenge for existing technologies. The gap between intent and execution: There is a lack of stable and reliable technical paths to translate a user's vague creative intent (such as "make a fast-paced trailer") into a series of precise and error-free atomic operation instructions on the timeline of professional editing software.

[0005] In conclusion, there is an urgent need to develop a new automated video editing solution to solve the remaining technical challenges in the field of automated video editing. Summary of the Invention

[0006] The purpose of this invention is to provide an automated video editing method and related apparatus based on multi-agent collaboration, so as to solve the technical problems that still exist in the existing technical solutions in terms of processing long videos, responding to personalized needs, and end-to-end automated execution capabilities.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides an automated video editing method based on multi-agent collaboration, comprising: The main control agent receives video editing requests input by the user in natural language. Based on the video editing requirements, the main control agent calls an independent multimodal analysis agent to perform a summary analysis of the media material to be processed, and receives the structured analysis results returned by the multimodal analysis agent. The master control agent integrates video editing requirements and structured analysis results to perform editing planning, generating an ordered execution sequence consisting of a series of atomic editing tool calls. The master control agent executes the ordered execution sequence in sequence, calling the atomic editing toolset to construct a complete editing project file. The rendering engine renders the editing project file and outputs the target video file.

[0008] A further improvement to the technical solution of this invention lies in that the step of receiving video editing requests input by the user in natural language form through the main control intelligent agent specifically includes: First, receive the user's original video editing request in natural language. Then, the main control agent analyzes the original video editing requirements to identify the material files to be processed and the core creative intent; wherein, the core creative intent is used as the subsequent video editing requirements.

[0009] A further improvement to the technical solution of this invention lies in the following steps: Based on video editing requirements, the main control agent calls an independent multimodal analysis agent to perform a summary analysis of the media material to be processed, and receives the structured analysis results returned by the multimodal analysis agent. Specifically, these steps include: For video footage, the multimodal analysis agent performs macro-level overview analysis, samples the video, submits the sampled frames to the visual language model, and generates a text summary describing the overall content, scene, style, and rhythm of the video. For audio materials, the multimodal analysis agent calls the audio processing model to analyze the music style and emotional tone, extract the beat sequence, and extract accurate information of the required time points; For image materials, the multimodal analysis agent calls the image recognition model to extract content tags and descriptions; The multimodal analysis agent integrates all analysis results into structured summary information and returns it to the master agent.

[0010] A further improvement to the technical solution of this invention lies in the fact that, according to video editing requirements, the step of calling an independent multimodal analysis agent through the main control agent to perform a summary analysis of the media material to be processed, and receiving the structured analysis results returned by the multimodal analysis agent, further includes: If the complexity of the video footage exceeds a preset threshold, a video deep understanding agent is invoked; the deep analysis tasks performed by the video deep understanding agent include: Receive analysis instructions and target video file identifiers issued by the main control agent; In the target video file, according to the analysis instructions, a two-stage strategy is adopted to obtain a high-density sampled frame sequence. In the two-stage strategy, the candidate time interval is first located by using the existing overview information or fast scanning, and then high-density sampling is performed at preset time intervals within the candidate time interval to obtain a high-density sampled frame sequence. The high-density sampled frame sequence is submitted to the visual language model for direct understanding in order to determine the precise start and end time codes of the target event and obtain precise segment information that meets the instruction requirements. All the precise fragments of information that meet the instructions are organized into a structured JSON object and returned to the master intelligent agent.

[0011] A further improvement of the technical solution of the present invention is that the complexity of the video material is determined based on one or more of the following: technical complexity, content complexity, artistic complexity, and post-processing complexity. The technical complexity is determined by one or more of the following: resolution, frame rate, encoding, compression, dynamic range, and color depth. The content complexity is determined by one or more of the following: number of scene elements, motion complexity, and visual effect complexity. The artistic complexity is determined by one or more of the following: composition complexity, narrative rhythm complexity, and emotional atmosphere complexity. The post-processing complexity is determined by one or more of the following: number of editing tracks, number of transitions, color grading complexity, and audio processing complexity.

[0012] A further improvement to the technical solution of this invention lies in the fact that the step of integrating video editing requirements and structured analysis results through a master control agent and performing editing planning to generate an ordered execution sequence consisting of a series of atomic editing tool calls specifically includes: By integrating video editing requirements and structured analysis results through the main control agent, all material clips and their precise timecodes for final editing are selected; The main control agent plans the number and hierarchy of video and audio tracks based on the editing logic; and generates one or more atomic editing tool call instructions for each selected material segment. The master control agent arranges all the calling instructions in logical order to form the final ordered execution sequence.

[0013] A further improvement to the technical solution of this invention lies in the step of integrating video editing requirements and structured analysis results through a master control agent and performing editing planning to generate an ordered execution sequence consisting of a series of atomic editing tool calls. If the video editing requirements include text or subtitles, the controlling agent will generate corresponding text addition tool call instructions.

[0014] A second aspect of the present invention provides an automated video editing system based on multi-agent collaboration, comprising: The requirement acquisition module is used to receive video editing requirements input by users in natural language through the main control agent; The analysis module is used to call an independent multimodal analysis agent through the main control agent to perform a summary analysis of the media material to be processed according to the video editing requirements, and to receive the structured analysis results returned by the multimodal analysis agent. The editing module integrates video editing requirements and structured analysis results through a master control agent and performs editing planning to generate an ordered execution sequence consisting of a series of atomic editing tool calls. The master control agent executes the ordered execution sequence in sequence, calling the atomic editing toolset to construct a complete editing project file. The rendering engine then renders the editing project file and outputs the target video file.

[0015] In a third aspect, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the automated video editing method based on multi-agent collaboration as described in any one of the first aspects of the present invention.

[0016] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the automated video editing method based on multi-agent collaboration as described in any one of the first aspects of the present invention.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention discloses an automated video editing method based on multi-agent collaboration, achieving a truly "unmanned" automated editing workflow. Users only need to input their creative ideas through natural language, and the method can independently complete the entire process from material understanding (interpretably, the media material to be processed can be input by the user or automatically invoked as needed), artistic conception, editing planning to final video output, generating a complete video editing effect that meets the user's input requirements in one go; and this capability is not limited by the length of the material (whether it is a short video, medium video, or long video) or the application scenario (such as movie trailers, product promotions, life records, etc.). Explaining the principle, the technical solution of this invention decomposes the complex editing task into multiple stages such as perception, planning, and execution, which are completed by agents with different roles working together. It also proposes specific mechanisms such as context isolation (only returning key structured information to the master agent) and atomic tool invocation, effectively solving the technical difficulties of existing technologies in processing long videos, responding to personalized needs, and end-to-end automated execution capabilities. In summary, the technical solution of this invention, through the design of a collaborative architecture of a master control agent, a multimodal analysis agent, and an atomic editing toolset, successfully deconstructs and modularizes complex video editing tasks. This architecture, through a context isolation mechanism, allows the heavy computation of multimodal analysis (especially complex video analysis) to be completed within a dedicated agent, returning only key structured information to the master control agent. This effectively solves the context window limitation problem of large language models when processing massive amounts of material information. At the same time, by atomizing and simplifying the editing operations, the difficulty for the master control agent to understand and plan complex editing sequences is significantly reduced, improving the stability and accuracy of system execution. This invention provides a new, efficient, and reliable technical path for achieving high-level, creative automated video editing. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an automated video editing method based on multi-agent collaboration, as described in an embodiment of the present invention. Figure 2 This is a schematic diagram of the principle framework in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the interaction between the main control intelligent agent and various modules throughout the entire process in an embodiment of the present invention; Figure 4This is a schematic diagram of a two-stage video analysis strategy in an embodiment of the present invention; Figure 5 This is an example diagram of the atomic editing toolset in an embodiment of the present invention; Figure 6 This is a schematic diagram of an automated video editing system based on multi-agent collaboration in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention; obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0021] Based on the technical solutions disclosed in the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0022] Please see Figure 1 The present invention provides an automated video editing method based on multi-agent collaboration, comprising the following steps: Step 1: Receive video editing requests from the user in natural language via the main control agent; Step 2: Based on the received video editing requirements, the main control agent calls an independent multimodal analysis agent to perform a summary analysis of the media material to be processed, and receives the structured analysis results returned by the multimodal analysis agent to achieve context space protection for the main control agent; in a specific exemplary technical solution, for complex video material, the main control agent calls a video deep understanding agent to obtain the structured information of the video intervals returned by the agent that describes the user's needs. Step 3: The main control agent integrates user requirements and the structured analysis results to perform editing planning, generating an ordered execution sequence consisting of a series of atomic editing tool calls. The main control agent executes the sequence in sequence, constructs a complete editing project file by calling the atomic editing toolset, and the rendering engine finally renders the project file and outputs it as the target video file, thus completing the end-to-end generation from text to video.

[0023] Unlike AIGC technology that creates content from scratch, the technical solution of this invention focuses on the automated editing of users' existing large-scale media assets. Through context isolation mechanisms, a two-stage video analysis strategy, and atomic tool design, it solves the core challenges of general models in understanding long videos, accurate positioning, and complex operation execution. It achieves an end-to-end process from natural language requirements to professionally edited content, demonstrating unique application value in the fields of intelligent editing for secondary creation and efficient utilization of existing materials, and in video editing where high authenticity of video content is required. The technical solution of this invention can deeply understand users' personalized creative needs and automate complex editing tasks throughout the entire process, completely freeing users from tedious operations and achieving a one-step generation from textual intent to the final video.

[0024] This invention provides an automated video editing method based on multi-agent collaboration. By constructing a three-layer collaborative architecture consisting of a master agent, a multimodal analysis agent, and an atomic editing toolset, it achieves end-to-end automated processing of user natural language requirements. The master agent is responsible for top-level planning, the multimodal analysis agent is responsible for deep understanding of the content through context isolation mechanisms, and the atomic editing toolset is responsible for the precise execution of low-level editing operations. The technical solution disclosed in this invention effectively integrates the reasoning capabilities of large-scale language models with the multimodal perception capabilities of specialized models, achieving efficient, accurate, and intelligent video editing.

[0025] Please see Figure 2 and Figure 3 The present invention provides an automated video editing method based on multi-agent collaboration, comprising the following steps: S1. The main control agent receives the video editing request input by the user in natural language, performs preliminary analysis of the request, and identifies the source files to be processed and the core creative intent.

[0026] In a specific exemplary technical solution, step S1 may also introduce an optional requirement clarification agent module. When the main control agent detects ambiguity or uncertainty in the natural language requirement input by the user, it will proactively invoke this module. The requirement clarification agent asks questions to the user through multi-turn dialogue, guiding the user to clarify their creative intent, and returns the clarified, structured requirements to the main control agent. This mechanism ensures that all subsequent automated operations are based on an accurate understanding of the user's intent, improving the satisfaction of the final product.

[0027] S2. Based on the creative intent, the master control agent calls the multimodal analysis agent once or multiple times to analyze the specified media material. This analysis process is completed inside the multimodal analysis agent, and the analysis results are returned in the form of structured data (such as a JSON object containing timecode and description).

[0028] S3. The master control agent integrates the user's requirements and the information returned in step S2, and decides whether to call the video deep understanding agent for complex video materials. The called video deep understanding agent performs high-cost deep analysis internally (e.g., two-stage analysis) and returns the filtered, structured, and precise segment information to the master control agent.

[0029] S4: The master control agent integrates user requirements and structured analysis results from multiple agents, performs editing planning, and generates an ordered execution sequence consisting of a series of atomic editing tool calls.

[0030] S5: The master control agent executes the sequence generated in S4 in sequence. By calling the atomic editing toolset, it completes operations such as track creation, material placement, and text addition on the virtual timeline, and finally builds a complete editing project file.

[0031] S6. The system outputs the editing project file generated in S5, completing the automated editing process.

[0032] The technical solution of this invention effectively solves core challenges in the field of automated video editing, such as multimodal collaborative understanding, long video context management, and complex operation logic execution, through an innovative multi-agent collaborative architecture. This invention truly enables fully automated video editing operations applicable to any scenario and duration, based solely on user text input—a capability not possessed by existing technologies and unreachable by current AIGC technologies. The context isolation, two-stage analysis, and atomized simplification of editing operations proposed in this invention not only improve the intelligence level and execution accuracy of automated editing but also provide a valuable system design paradigm for handling other complex, multi-stage AI-driven tasks, showing broad application prospects in content creation, short video marketing, and personalized video generation.

[0033] In a preferred embodiment of the present invention, step S2 specifically comprises: S201. For video footage, the multimodal analysis agent performs a macro-level overview analysis, such as sampling the video at second-level intervals, submitting the sampled frames to the visual language model, and generating a text summary describing the overall content, scene, style, and rhythm of the video. Interpretably, this step aims to provide the controlling agent with rapid, global decision-making support, rather than precise time points.

[0034] S202. For audio materials, the multimodal analysis agent calls the audio processing model to analyze its musical style and emotional tone, extract the beat sequence, extract accurate information of the required time points, etc.

[0035] S203. For image materials, the multimodal analysis agent calls the image recognition model to extract their content tags and descriptions.

[0036] S204. The multimodal analysis agent integrates all analysis results into a structured summary (e.g., a JSON object containing a summary and tags) and returns it to the master agent.

[0037] In a preferred embodiment of the present invention, the analysis task of multimodal materials is isolated within one or more dedicated multimodal analysis agents, and the analysis task of complex videos is isolated within video deep understanding agents, thereby achieving context space protection for the master agent. Specifically, the master agent only needs to convey the user's request (such as "find all sunset shots in the video") to the multimodal video deep understanding agents. The analysis agents internally complete the frame-by-frame or block-by-block processing of the video and return only structured data containing timecodes and content descriptions. This mechanism avoids the influx of massive amounts of low-information-density raw frame descriptions into the context of the master agent, allowing it to reserve sufficient computing resources for higher-level creative planning and logical reasoning.

[0038] In a preferred embodiment of the present invention, in step S3, the deep analysis task performed by the invoked video deep understanding agent may include: S301: Receive the specific analysis instructions and target video file identifier issued by the main control agent.

[0039] S302. Perform a precise microscopic analysis in the specified video file; for example, according to the instruction "find the segment where the cat appears", a two-stage strategy is adopted: first, use the existing overview information or a quick scan to locate the candidate time intervals that may contain the cat, and then perform high-density sampling in these intervals at time intervals of 0.1 seconds or less.

[0040] S303. Submit the high-density sampled frame sequence to the visual language model for direct understanding in order to determine the precise start and end timecodes of the target event.

[0041] S304. Organize all found precise fragment information (including timecode and detailed description) that meets the instruction requirements into a structured JSON object and return it to the master intelligent agent.

[0042] In a preferred embodiment of the present invention, a two-stage analysis strategy is employed in the video deep understanding module for video materials, balancing analysis efficiency and accuracy. The first stage is a macro-level overview analysis, which quickly obtains the overall content summary and scene distribution of the video through low-density sampling (e.g., one frame per second). The second stage is a micro-level precise analysis, which performs high-density sampling (e.g., one frame per 0.1 seconds) within the candidate time interval determined by the controlling agent or the overview analysis to obtain the precise time point of the event. This strategy effectively avoids costly frame-by-frame analysis of the entire video, significantly improving processing speed. For example, as shown... Figure 4 As shown.

[0043] In a preferred embodiment of the present invention, in step S4, the main control agent performs editing planning, as detailed below: S401, The main control agent integrates the multimodal information from step S2 and the possible deep understanding precise fragment information from step S3.

[0044] S402. Based on user needs and all collected information, select all material clips and their precise timecodes for final editing.

[0045] S403, the main control agent plans the number and hierarchy of video and audio tracks according to the editing logic.

[0046] S404: The master control agent generates one or more atomic editing tool call instructions for each selected segment, including parameters such as specifying the material name, the start and end times on the timeline, and the start and end times in the source material.

[0047] S405. If the user's request includes text or subtitles, the main control agent will generate the corresponding text addition tool call command.

[0048] S406 The master control agent arranges all calling instructions in a logical order (such as creating tracks first and then adding segments) to form the final execution sequence.

[0049] In a preferred embodiment of this invention, complex editing operations are decomposed into a series of atomic editing tools with single functions and clearly defined interfaces. Simultaneously, a "no-modification strategy" is adopted, eliminating the complex state management associated with deletion and modification operations. For example, editing behavior is decomposed into basic addition operations such as "creating tracks" and "adding clips." The controlling agent only needs to call these preset atomic tools to construct complex editing projects. This design simplifies the controlling agent's task from learning complex editing software logic to calling and arranging a limited set of tools, significantly reducing the difficulty of understanding and execution.

[0050] In a preferred embodiment of the present invention, in step S5, the atomic editing toolset includes, but is not limited to: `add_track`: Creates a track. This tool allows you to specify the track type (such as a video track or an audio track) via parameters to support the placement of different types of content.

[0051] `add_clip`: Adds a clip to the specified timeline. This tool is a highly versatile interface that defines various clip attributes through parameters, such as: the clip type (video, audio, or text), the referenced source material, its start and end times in the source material, its placement on the timeline, and its text content. This tool allows you to add various elements, such as video, audio, and text, as a single "clip" to the appropriate timeline. For example, as... Figure 5 As shown.

[0052] The technical solution provided by this invention, through an innovative context isolation mechanism, enables the intelligent agent, as the core of the planning process, to handle video footage of arbitrary length. This solves the context window limitation problem of large language models. Users only need to provide text input, and the system can automatically and end-to-end complete the editing of long videos without human intervention. Unlike existing template-based tools that are limited to specific video editing scenarios, the multi-agent collaborative architecture and deep content understanding capabilities of this invention enable it to handle video footage of any scene and subject matter. Whether editing highlights from a feature film or creating a short social media video of a few dozen seconds, the method of this invention can perform high-quality automated editing based on user text requirements, exhibiting extremely high versatility and flexibility. The two-stage video analysis strategy adopted by this invention combines macro-level overview with micro-level precise positioning, enabling it to efficiently find precise segments from massive amounts of video data that meet the user's fuzzy and abstract creative needs, achieving a truly deep understanding of the video content. This invention significantly reduces the difficulty and error rate of the main control agent translating high-level creative intentions into low-level software operations by abstracting complex editing operations into an atomized and extremely simplified toolset. It builds a solid bridge from abstract ideas to precise execution, enabling machines to stably perform "creative" work.

[0053] In one embodiment of the present invention, an automated video editing system based on multi-agent collaboration is provided. This system can be used to implement the above-mentioned method. Specifically, the automated video editing system includes: a master control agent module, a multimodal analysis module, a video deep understanding module, and an editing execution module.

[0054] in: The master control agent module is responsible for receiving and parsing the user's natural language requirements, performing top-level task planning and scheduling, distributing analysis tasks to other agents, and generating the final editing execution sequence based on the returned results.

[0055] The multimodal analysis module contains one or more multimodal analysis agents, which are responsible for executing the multimodal material analysis tasks issued by the master agent and providing macro-level decision-making information to the master agent.

[0056] The video deep understanding module contains one or more video deep understanding agents that can be independently invoked by the master control agent. These agents are responsible for performing high-precision deep analysis on single, complex video materials according to the specific instructions of the master control agent, and returning refined and accurate segment information.

[0057] The editing execution module contains a set of atomic, incrementally simple editing tool APIs, which are responsible for receiving and executing instructions generated by the master intelligent agent to complete the underlying operations of the virtual editing project.

[0058] This invention presents an automated video editing method and system based on multi-agent collaboration. Through innovative architectural design, it effectively deconstructs complex editing tasks, solves the challenge of processing massive amounts of video information through context isolation and a two-stage analysis strategy, and reduces the complexity of AI execution through operation atomization. Ultimately, it achieves a highly efficient, high-precision, and highly intelligent automated video editing solution with significant application value and commercial prospects in the content creation industry. This invention successfully bridges the gap between abstract creative concepts and specific editing operations through the planning capabilities of the master agent. Users can input high-level, emotionally and stylistically descriptive instructions such as "create a suspenseful trailer with fast-paced music during the climax," rather than just precise instructions like "cut from the 5th to the 10th second." The master agent can understand these abstract concepts and decompose them into a series of precise, executable, atomic tool calls, such as "identify key conflict shots," "select highlight moments," "match fast-paced background music," and "quickly switch at key points." This transformation capability is lacking in traditional tools and template-based applications, enabling machines to perform truly "creative" work.

[0059] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the apparatus embodiments, please refer to the embodiments of the method of the present invention.

[0060] Please see Figure 6 In this embodiment of the invention, an automated video editing system based on multi-agent collaboration is provided, comprising: The requirement acquisition module is used to receive video editing requirements input by users in natural language through the main control agent; The analysis module is used to call an independent multimodal analysis agent through the main control agent to perform a summary analysis of the media material to be processed according to the video editing requirements, and to receive the structured analysis results returned by the multimodal analysis agent. The editing module integrates video editing requirements and structured analysis results through a master control agent and performs editing planning to generate an ordered execution sequence consisting of a series of atomic editing tool calls. The master control agent executes the ordered execution sequence in sequence, calling the atomic editing toolset to construct a complete editing project file. The rendering engine then renders the editing project file and outputs the target video file.

[0061] In one embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used to execute operations of an automated video editing method based on multi-agent collaboration.

[0062] In one embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM (Random Access Memory) or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the automated video editing method based on multi-agent cooperation in the above embodiments.

[0063] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.

[0064] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An automated video editing method based on multi-agent collaboration, characterized in that, include: The main control agent receives video editing requests input by the user in natural language. Based on the video editing requirements, the main control agent calls an independent multimodal analysis agent to perform a summary analysis of the media material to be processed, and receives the structured analysis results returned by the multimodal analysis agent. The master control agent integrates video editing requirements and structured analysis results to perform editing planning, generating an ordered execution sequence consisting of a series of atomic editing tool calls. The master control agent executes the ordered execution sequence in sequence, calling the atomic editing toolset to construct a complete editing project file. The rendering engine renders the editing project file and outputs the target video file.

2. The automated video editing method based on multi-agent collaboration according to claim 1, characterized in that, The specific steps involved in receiving video editing requests from users in natural language input via the main control agent include: First, receive the user's original video editing request in natural language. Then, the main control agent analyzes the original video editing requirements to identify the material files to be processed and the core creative intent; wherein, the core creative intent is used as the subsequent video editing requirements.

3. The automated video editing method based on multi-agent collaboration according to claim 1, characterized in that, Based on video editing requirements, the steps of calling an independent multimodal analysis agent through the main control agent to perform a summary analysis of the media material to be processed, and receiving the structured analysis results returned by the multimodal analysis agent specifically include: For video footage, the multimodal analysis agent performs macro-level overview analysis, samples the video, submits the sampled frames to the visual language model, and generates a text summary describing the overall content, scene, style, and rhythm of the video. For audio materials, the multimodal analysis agent calls the audio processing model to analyze the music style and emotional tone, extract the beat sequence, and extract accurate information of the required time points; For image materials, the multimodal analysis agent calls the image recognition model to extract content tags and descriptions; The multimodal analysis agent integrates all analysis results into structured summary information and returns it to the master agent.

4. The automated video editing method based on multi-agent collaboration according to claim 3, characterized in that, According to the video editing requirements, the steps of calling an independent multimodal analysis agent through the main control agent to perform a summary analysis of the media material to be processed, and receiving the structured analysis results returned by the multimodal analysis agent, further include: If the complexity of the video footage exceeds a preset threshold, a video deep understanding agent is invoked; the deep analysis tasks performed by the video deep understanding agent include: Receive analysis instructions and target video file identifiers issued by the main control agent; In the target video file, according to the analysis instructions, a two-stage strategy is adopted to obtain a high-density sampled frame sequence. In the two-stage strategy, the candidate time interval is first located by using the existing overview information or fast scanning, and then high-density sampling is performed at preset time intervals within the candidate time interval to obtain a high-density sampled frame sequence. The high-density sampled frame sequence is submitted to the visual language model for direct understanding in order to determine the precise start and end time codes of the target event and obtain precise segment information that meets the instruction requirements. All the precise fragments of information that meet the instructions are organized into a structured JSON object and returned to the master intelligent agent.

5. The automated video editing method based on multi-agent collaboration according to claim 4, characterized in that, The complexity of video footage is determined based on one or more of the following: technical complexity, content complexity, artistic complexity, and post-processing complexity. The technical complexity is determined by one or more of the following: resolution, frame rate, encoding, compression, dynamic range, and color depth. The content complexity is determined by one or more of the following: number of scene elements, motion complexity, and visual effect complexity. The artistic complexity is determined by one or more of the following: composition complexity, narrative rhythm complexity, and emotional atmosphere complexity. The post-processing complexity is determined by one or more of the following: number of editing tracks, number of transitions, color grading complexity, and audio processing complexity.

6. The automated video editing method based on multi-agent collaboration according to claim 1, characterized in that, The steps of integrating video editing requirements and structured analysis results through a master control agent and performing editing planning to generate an ordered execution sequence consisting of a series of atomic editing tool calls specifically include: By integrating video editing requirements and structured analysis results through the main control agent, all material clips and their precise timecodes for final editing are selected; The main control agent plans the number and hierarchy of video and audio tracks based on the editing logic; and generates one or more atomic editing tool call instructions for each selected material segment. The master control agent arranges all the calling instructions in logical order to form the final ordered execution sequence.

7. The automated video editing method based on multi-agent collaboration according to claim 6, characterized in that, In the step of integrating video editing requirements and structured analysis results through a master control agent and performing editing planning to generate an ordered execution sequence consisting of a series of atomic editing tool calls, If the video editing requirements include text or subtitles, the controlling agent will generate corresponding text addition tool call instructions.

8. An automated video editing system based on multi-agent collaboration, characterized in that, include: The requirement acquisition module is used to receive video editing requirements input by users in natural language through the main control agent; The analysis module is used to call an independent multimodal analysis agent through the main control agent to perform a summary analysis of the media material to be processed according to the video editing requirements, and to receive the structured analysis results returned by the multimodal analysis agent. The editing module integrates video editing requirements and structured analysis results through a master control agent and performs editing planning to generate an ordered execution sequence consisting of a series of atomic editing tool calls. The master control agent executes the ordered execution sequence in sequence, calling the atomic editing toolset to construct a complete editing project file. The rendering engine then renders the editing project file and outputs the target video file.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the automated video editing method based on multi-agent collaboration as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the automated video editing method based on multi-agent collaboration as described in any one of claims 1 to 7.

Images