Agent-based video clip method, apparatus and product

Abstract

The present disclosure provides an agent-based video clip method and device, electronic equipment, storage medium and computer program product, which are related to the technical field of artificial intelligence, specifically related to the technical fields of agent, large model, video clip, and can be applied to the video clip scene. The specific implementation scheme is: a plurality of progressive shot splitting algorithms are used to split the shots in the video to be clipped to obtain a shot set; a script generation tool is called by an agent to generate a shot script that meets the video clipping requirements, and the shot script represents the splicing order of a plurality of target shots selected from the shot set; a video splicing tool is called by the agent to splice a plurality of target shots according to the shot script to generate a clipped video. The present disclosure improves the shot accuracy based on the plurality of progressive shot splitting algorithms, and generates a shot script and performs video clipping in combination with the agent and related tools, thereby improving the video clipping efficiency and the coherence of the edited video.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically to the technical fields of intelligent agents, large models, and video editing, and in particular to a video editing method, device, electronic device, storage medium, and computer program product based on intelligent agents, which can be applied to video editing scenarios. Background Technology

[0002] In short video applications, users generally have a need for independent video editing and content creation. Currently, video editing is typically done manually. Users manually select video footage and splice video clips to obtain the edited video. Manual editing is inefficient and results in poor video coherence. Summary of the Invention

[0003] This disclosure provides a video editing method, apparatus, electronic device, storage medium, and computer program product based on intelligent agents.

[0004] According to the first aspect, an agent-based video editing method is provided, comprising: splitting the video to be edited into segments using a progressive multi-segment splitting algorithm to obtain a segment set; generating a segment script that meets the video editing requirements by calling a script generation tool through an agent, wherein the segment script represents the splicing order of multiple target segments that meet the video editing requirements in the segment set; and splicing multiple target segments according to the segment script by calling a video splicing tool through an agent to generate the edited video.

[0005] According to a second aspect, an agent-based video editing device is provided, comprising: a storyboard splitting unit configured to split the storyboards in the video to be edited using a progressive multi-storyboard splitting algorithm to obtain a storyboard set; a script generation unit configured to generate a storyboard script that meets the video editing requirements by calling a script generation tool through an agent, wherein the storyboard script represents the splicing order of multiple target storyboards that meet the video editing requirements in the storyboard set; and a video editing unit configured to splice multiple target storyboards according to the storyboard script by calling a video splicing tool through an agent to generate an edited video.

[0006] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method as described in any implementation of the first aspect.

[0007] According to a fourth aspect, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to perform the method described in any implementation of the first aspect.

[0008] According to a fifth aspect, a computer program product is provided, comprising: a computer program that, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0009] According to the technology disclosed herein, a video editing method and apparatus based on an intelligent agent are provided. The method involves splitting the video to be edited into a set of scenes using a progressive multi-scene splitting algorithm; generating a scene script that meets the video editing requirements by having the intelligent agent invoke a script generation tool; and generating an edited video by having the intelligent agent invoke a video splicing tool to splice the multiple target scenes according to the scene script. This provides an automated video editing method. The progressive multi-scene splitting algorithm improves the accuracy of scene splitting, and the combination of the intelligent agent and related tools to generate scene scripts and execute video editing improves video editing efficiency and the coherence of the edited video.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is an exemplary system architecture diagram that can be applied to an embodiment of this disclosure; Figure 2 This is a flowchart of an embodiment of the agent-based video editing method according to the present disclosure; Figure 3 This is a schematic diagram illustrating an application scenario of the agent-based video editing method according to this embodiment; Figure 4 This is a flowchart of yet another embodiment of the agent-based video editing method according to the present disclosure; Figure 5 This is a structural diagram of an embodiment of the agent-based video editing apparatus according to the present disclosure; Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] The technical solutions disclosed herein involve the collection, storage, use, processing, transmission, provision, and disclosure of various types of information, such as user personal information, in accordance with relevant laws and regulations and do not violate public order and good morals.

[0014] Figure 1 An exemplary architecture 100 is shown for which the agent-based video editing method and apparatus of this disclosure can be applied.

[0015] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The communication connections between terminal devices 101, 102, and 103 form a network topology. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0016] Terminal devices 101, 102, and 103 can be hardware or software that supports network connectivity for data acquisition, interaction, and processing. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices that support network connectivity, information acquisition, interaction, display, and processing functions, including but not limited to smartphones, tablets, e-book readers, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are imposed here.

[0017] Server 105 can be a server that provides various services, such as acquiring videos to be edited sent by terminal devices 101, 102, and 103, using a progressive multi-scene splitting algorithm to split the scenes in the videos to be edited, and performing video editing through an intelligent agent. As an example, server 105 can be a cloud server.

[0018] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (such as software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0019] It should also be noted that the agent-based video editing method provided in the embodiments of this disclosure is generally executed by a server, but the possibility of it being executed by a terminal device, or by the server and the terminal device cooperating with each other, is not excluded. Accordingly, the various parts (e.g., various units) included in the agent-based video editing device can be all located in the server, all located in the terminal device, or separately located in the server and the terminal device.

[0020] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Any number of terminal devices, networks, and servers can be included depending on implementation needs. When the electronic devices on which the agent-based video editing method runs do not require data transmission with other electronic devices, the system architecture may consist only of the electronic devices (e.g., servers or terminal devices) on which the agent-based video editing method runs.

[0021] Please refer to Figure 2 , Figure 2 A flowchart of an agent-based video editing method provided in this disclosure embodiment. Flowchart 200 includes the following steps: Step 201: Use a progressive multi-scene splitting algorithm to split the scenes in the video to be edited to obtain a set of scenes.

[0022] In this embodiment, the executing entity of the agent-based video editing method (e.g., Figure 1 The server in the video can use a progressive multi-scene splitting algorithm to split the scenes in the video to be edited, and obtain a set of scenes.

[0023] The videos to be edited are various original video materials adapted for both short and long video scenarios. Taking short video scenarios as an example, the videos to be edited include, but are not limited to: product promotion short videos (such as beauty and digital product promotional videos); knowledge popularization short videos (such as industry tips and tutorial videos); lifestyle sharing short videos (such as daily vlogs and scene recording videos); and commercial promotion short videos (such as brand promotion and event promotional videos).

[0024] Storyboards refer to video segments formed by breaking down a complete video to be edited according to certain logic. These segments have independent themes and content boundaries and can be used as independent editing units for subsequent splicing and editing. Their content is coherent and themes are unified, and they can independently present complete information segments.

[0025] For example, a video introducing "product usage steps" can be broken down into three shots: ① Product unboxing scene (only showing the unboxing action, without any extra scenes), ② Product function demonstration scene (only showing the function operation process), ③ Product usage effect scene (only showing the finished product after use); another example is a lifestyle vlog video, which can be broken down into ① a scene of leaving home, ② a scene of visiting a store, and ③ a scene of returning home. Each shot is an independent and editable unit with no overlapping scenes or redundant content.

[0026] Segmentation splitting algorithms are algorithms used to identify frame boundaries and scene transition nodes in a video to be edited, and to split the complete video into multiple independent segment segments. The core function is to achieve structured splitting of video frames, providing operable independent units for subsequent editing, without limiting the splitting accuracy or the number of splits.

[0027] For example, scene recognition-based splitting algorithms (identifying splitting nodes through changes in screen content and camera transitions) and frame similarity-based splitting algorithms (determining splitting nodes by comparing features of adjacent frames) are both types of scene splitting algorithms, which can split the original video into scenes.

[0028] The progressive multi-scene splitting algorithm refers to the use of multiple scene splitting algorithms applied sequentially to the video to be edited, gradually optimizing the scene splitting accuracy. First, a preliminary scene splitting is completed, and then further refined through subsequent splitting algorithms, ultimately resulting in a series of algorithm combinations that meet the editing requirements. Each algorithm is executed sequentially and progressively, without skipping any splitting steps, ensuring the completeness and accuracy of the scene splitting.

[0029] As an example, two different processing logics are used to perform shot splitting and boundary optimization in sequence. First, the first type of shot splitting is performed on the complete video to be edited. From a global perspective, obvious shot transitions, scene changes, and content theme changes are identified in the video. Based on content relevance and visual continuity, the complete video is divided into several continuous video segments, completing the initial shot segmentation.

[0030] Secondly, based on the video segments obtained from the initial segmentation, a second type of progressive segmentation process is performed. This process identifies and optimizes the start and end boundaries of each segment from a local detail perspective, identifies natural shot boundaries within the segments, and corrects issues such as blurred boundaries and redundant ranges in the initial segmentation. For segments with complete content and consistent shot ranges, the original segment range is maintained without modification. Finally, all video segments that have undergone two levels of progressive segmentation and boundary optimization are compiled and organized to form a structured storyboard set.

[0031] As another example, this approach utilizes two different types of segmentation rules based on visual characteristics, employing a step-by-step process with clear functional distinctions to independently achieve scene segmentation. First, the first type of segmentation rule is used, based on macroscopic visual style and scene environmental characteristics. It focuses on significant changes in background environment, overall tone, and scene space, using these as the basis for segmenting video content into several initial video clips with unified style and scene, achieving initial segmentation based on the scene dimension. Second, the progressive second type of segmentation rule is executed, based on the main subject and shot expression characteristics for secondary segmentation. It focuses on changes in the main subject, shooting angle, and camera movement, further distinguishing different shot expression units within the initial scene-unified clips. For clips with simple shot expression and coherent content, the complete clips are retained without repeated segmentation. Finally, all independent video clips after the two-level progressive feature segmentation are integrated and uniformly organized into a standardized storyboard set for subsequent video editing.

[0032] It should be noted that this embodiment does not limit the number of progressive segmentation algorithms, and the number of segmentation algorithms can be flexibly set according to the actual situation.

[0033] In some optional implementations of this embodiment, the execution entity performs step 201 as follows to obtain a storyboard set: First step, the video to be edited is split into multiple preliminary storyboards using a first preset splitting algorithm; Second step, the multiple preliminary storyboards are split into multiple preliminary storyboards using a second preset splitting algorithm to obtain a storyboard set.

[0034] The first pre-defined segmentation algorithm is, for example, a global shot change detection algorithm (such as the scenedetect algorithm), which relies on the overall image differences between frames of the entire video segment and the features of large-scale scene environment changes to complete a large-scale, coarse-grained segmentation of shots. The second pre-defined segmentation algorithm is, for example, a local shot boundary recognition algorithm, which focuses on the changes in the main subject of the image and the details of shot movement and switching within a single segment, and performs a finer secondary segmentation of the existing segments.

[0035] As an example, the first preset splitting algorithm is applied to the entire video to be edited. This algorithm identifies scene changes and large-scale shot transitions from a global perspective. Based on the overall relevance of the content, the complete video to be edited is divided into several content segments, forming multiple preliminary storyboards with broad boundaries. Subsequently, each preliminary storyboard is processed independently and input into the second preset splitting algorithm, focusing on subtle shot transitions and changes in content within the segment, further refining the internal boundaries of the preliminary storyboards. After all the secondary splitting is completed, all the resulting video segments are aggregated and integrated to form a complete storyboard set.

[0036] In this implementation, a two-level, multi-dimensional splitting algorithm is used to progressively process the scene, taking into account both the global scene division and the refinement of local shots. This improves the comprehensiveness and rationality of the scene division, providing a regular and complete foundation for subsequent video editing.

[0037] In some optional implementations of this embodiment, the execution entity may also perform the following operations: First, determine whether a scene in the scene set is a single scene using a large artificial intelligence model; then, in response to the determination that it is not a single scene, delete the scenes in the scene set that are not single scenes.

[0038] Large-scale AI models are pre-trained models with multimodal content understanding, semantic analysis, and logical discrimination capabilities. They can autonomously identify and objectively judge the overall semantics and shot structure of video clips, outputting standardized judgment results. Examples include large language models and multimodal large-scale models.

[0039] A single shot refers to the smallest video shot unit that has a continuous shooting perspective, a unified theme in the content, no internal shot switching or scene jumps, and maintains a complete and independent shot structure.

[0040] As an example, after forming the storyboard set, an AI model is invoked as an independent judgment subject. Each storyboard segment within the set is sequentially input into the model. The model comprehensively identifies and judges the shot continuity, scene transition features, and scene changes of individual storyboards, verifying whether each storyboard conforms to the shot structure standards of a single storyboard. For non-single storyboards that are determined not to be single storyboards, or that contain multiple shot splices or scene transitions, a removal process is performed, retaining only single storyboards with complete structure and consistent shot composition. This results in an updated storyboard set, allowing subsequent steps to edit the video based on single storyboards, improving video editing quality.

[0041] In this implementation, the large model is used to accurately identify the shot structure, eliminate composite multi-shot segments, ensure that the remaining shot unit structure is uniform and standardized, reduce interference from invalid materials, and improve the regularity and adaptability of subsequent editing and arrangement.

[0042] In some optional implementations of this embodiment, the execution entity may also perform the following operation: filtering out sub-shots in the sub-shot set that do not conform to the preset filtering rules.

[0043] Preset filtering rules are pre-defined quantitative or content constraints used to screen and remove storyboard segments from the storyboard collection that do not meet basic video editing standards, content requirements, or final cut adaptation conditions. Preset filtering rules include, but are not limited to, duration threshold rules, image quality rules, content compliance rules, and effective image proportion rules; for example, removing fragmented storyboards that are too short, removing invalid storyboards with blurry or black screens, and filtering out redundant and meaningless segments.

[0044] As an example, for the storyboard set obtained in step 201 or the storyboard set that has undergone singleness verification based on a large model, preset filtering rules are used as a unified screening standard to traverse all storyboard units within the storyboard set. Each storyboard is compared and verified item by item in terms of duration parameters, visual state, content validity, etc., to determine whether a single storyboard meets the requirements of each preset filtering rule. For storyboards with excessive parameters, abnormal visuals, or substandard content, a filtering process is performed to remove them, retaining only valid storyboards that meet all the specifications, thus completing the standardization and purification of the storyboard set.

[0045] In this implementation, the storyboard set is screened and purified through a unified preset filtering rule, eliminating inferior, invalid and non-compliant segments, reducing the amount of effective material, ensuring the overall quality of subsequent editing materials, and improving video editing efficiency and the overall viewing experience of the final product.

[0046] In some optional implementations of this embodiment, the execution entity can perform the first step described above and split multiple preliminary scenes in the following manner: First, spatial features of video frames in the preliminary scenes are extracted using a two-dimensional convolutional network to obtain a spatial feature sequence; then, temporal features of the spatial feature sequence are extracted using a one-dimensional temporal convolutional network; then, the transition center point in the preliminary scenes is determined by combining the spatial features, temporal features, and similarity features between adjacent video frames in the preliminary scenes; finally, the preliminary scenes are split using the transition center point as the split boundary.

[0047] Two-dimensional convolutional networks are feature extraction networks for single-frame images. They rely on two-dimensional convolutional kernels to capture local pixel, texture, contour, scene and target spatial correlation information within a single video frame, thereby achieving feature encoding of static image content.

[0048] One-dimensional temporal convolutional networks are feature extraction networks for time-series sequences. They employ a one-dimensional convolutional structure to perform convolution operations on ordered feature sequences along the time dimension, thereby uncovering temporal changes, motion correlations, and dynamic evolution patterns between consecutive frames.

[0049] Spatial features characterize the static visual attributes and spatial structure information within a single video frame, reflecting the content composition and visual distribution of that frame. Examples include static visual features such as image texture, object outlines, background environment, color distribution, target location, and scene layout.

[0050] Temporal features characterize the dynamic changes and temporal relationships between consecutive video frames, reflecting the differences in the evolution of the scene over time. Examples include temporal change features such as the amplitude of object movement, changes in camera movement, gradual changes in content between frames, and dynamic fluctuations in the image.

[0051] The transition center point refers to the key time frame position within the initial shot where there is a significant shot change, scene change, or sudden change in the content of the scene. It serves as the core dividing boundary point for separating the different shot contents before and after.

[0052] As an example, consecutive video frames arranged in chronological order are extracted from the initial storyboard. Each frame image is input into a two-dimensional convolutional network. Through multi-layer two-dimensional convolution operations, the unique spatial features of each frame are extracted and arranged in chronological order to form a continuous spatial feature sequence.

[0053] The spatial feature sequence is input into a one-dimensional temporal convolutional network, and deep temporal modeling is performed along the time dimension to mine dynamic correlation information between frames and output the corresponding temporal features.

[0054] The similarity between adjacent video frames in the preliminary storyboard is calculated synchronously to obtain inter-frame similarity features. Inter-frame similarity features quantify the degree of similarity between adjacent consecutive video frames in the same preliminary storyboard in dimensions such as image content, pixel distribution, texture color, and subject structure. This objectively reflects the degree of content difference and abrupt changes in the image between consecutive frames. Spatial features, temporal features, and inter-frame similarity features are fused and correlated to comprehensively determine the location of abrupt changes in image content and accurately locate the transition center point within the preliminary storyboard.

[0055] Lock the frame position corresponding to the transition center point, set this key position as the fixed split boundary, and perform truncation and splitting on the complete preliminary storyboard to complete the shot splitting process for a single preliminary storyboard.

[0056] This implementation integrates multi-dimensional features of spatial vision, temporal dynamics, and inter-frame similarity to accurately locate key positions in shot transitions, improving the accuracy of shot splitting boundary determination and ensuring that the splitting results conform to the actual shot switching patterns.

[0057] In some optional implementations of this embodiment, the execution entity can determine the transition center point in the preliminary storyboard in the following way: combining spatial features, temporal features and similarity features, the transition center point is determined by the first prediction head, and the gradual transition interval is determined by the second prediction head. The gradual transition interval represents a continuous frame sequence of smooth transitions between adjacent storyboards.

[0058] The prediction head is a dedicated branch inference module mounted on the backend of the feature extraction network. Relying on the fused multi-dimensional feature information, it performs independent regression and classification predictions for different detection targets, and can output different types of key parameters and location information. The first prediction head is used to determine the transition center point, and the second prediction head is used to determine the gradual transition range.

[0059] A gradient transition range specifically refers to a continuous frame segment in which the image presents non-instantaneous switching effects such as fade-in, fade-out, dissolve, and slow gradation during the switching between adjacent shots. Within this range, the image content gradually transitions and the differences change continuously, which is different from the sudden change of a single frame. It is the temporal range of smooth transition between shots.

[0060] As an example, the fused spatial features, temporal features, and inter-frame similarity features are uniformly input into the dual-branch prediction structure. The first prediction head focuses on learning the rules of sudden shot changes, inferring and outputting accurate transition center point coordinates and temporal information; at the same time, the second prediction head identifies slow scene transitions and delineates the corresponding gradual transition range.

[0061] In this implementation, the aforementioned execution entity can split the initial storyboard using the transition center point as the split boundary in the following way: First, verify the transition center point based on the gradient transition interval; then, in response to the successful verification, split the initial storyboard using the transition center point as the split boundary.

[0062] As an example, using the temporal range and transition characteristics of the gradual transition interval as the verification benchmark, we check whether the transition center point is within a reasonable transition area and whether it conforms to the actual shot switching pattern. If the transition center point is determined to be valid and the position matching is reasonable after interval verification, the verification is confirmed to be passed. Using this transition center point as the split boundary, the preliminary accurate splitting of the shot is completed.

[0063] In this implementation, dual prediction heads are used to detect key parameters, and cross-validation is completed by combining the gradual transition interval. This avoids the problem of single-point false detection, improves the reliability of transition positioning, and further optimizes the rationality and accuracy of the segmentation boundary.

[0064] Step 202: The agent calls the script generation tool to generate a storyboard script that meets the needs of video editing.

[0065] In this embodiment, the aforementioned execution entity invokes a script generation tool through an intelligent agent to generate a storyboard script that meets the video editing requirements. The storyboard script represents the splicing order of multiple target storyboards in the storyboard set that meet the video editing requirements. The storyboard script is a standardized, structured arrangement file / instruction set output by the script generation tool, used to select several target storyboards from the formed storyboard set that fit the current video editing requirements, and to clearly define the sequential order and connection relationships of each target storyboard.

[0066] An intelligent agent is an intelligent program entity with autonomous task scheduling, tool invocation, instruction parsing, and logical decision-making capabilities. It can autonomously match and invoke corresponding functional tools according to business needs, complete a coherent task execution process according to predetermined goals, and has the ability to coordinate and schedule multiple tools and issue autonomous instructions.

[0067] Intelligent agents can be task-driven automated scheduling agents, content creation-specific scheduling agents, or video processing-specific operational agents; they can autonomously identify video editing targets, autonomously select the required functional tools and initiate call requests, and automatically complete tool interactions and task flows.

[0068] The script generation tool is an independently configured functional application module that can receive task instructions and basic material information, combine preset rules and content requirements, and autonomously arrange and output standardized executable editable scripts, providing a well-organized arrangement basis for video processing.

[0069] Script generation tools can be used as visual editing script generation modules, video arrangement instruction generators, and content combination rule compilation tools. They can automatically arrange segment combinations based on content preferences, final film style, narrative logic, and other conditions, outputting structured arrangement files. For example, script generation tools can be implemented based on large-scale artificial intelligence models.

[0070] Video editing requirements are pre-defined constraints and content creation requirements for the final video output, set by the user or business scenario. These include non-technical algorithm-based editing requirements such as narrative logic, final style, segment combination format, and content expression logic, which constrain the selection and arrangement of target scenes.

[0071] For example, to meet narrative logic requirements, video clips should be combined according to the narrative logic of content introduction, highlight display, and summary guidance; to meet style arrangement requirements, priority should be given to selecting atmospheric scenes and unifying the rhythm of the scenes to form a coherent visual style; to meet content structure requirements, the content hierarchy of the final product should be controlled, core main scenes should be retained first, and redundant and irrelevant content should be deleted; to meet scene adaptation requirements, the logic of segment connection should be optimized to ensure smooth content expression in order to adapt to online dissemination scenarios.

[0072] As an example, after the video to be edited is broken down into storyboards and a complete set of storyboards is formed, the scheduling agent enters the task execution state. The agent autonomously analyzes the current predetermined video editing requirements, reads the constraints such as the content style, narrative order, and content emphasis in the requirements; then the agent actively retrieves callable functional components, accurately calls the script generation tool, and simultaneously pushes the basic information of the storyboard set and the complete video editing requirements information to the script generation tool.

[0073] After receiving input information, the script generation tool, based on the constraints of editing requirements, traverses all storyboards in the storyboard set, selecting multiple target storyboards that match the content, style, and content expression requirements. Simultaneously, combining narrative logic and content expression order, it plans the sequential arrangement of each target storyboard, generating a standardized, structured storyboard script. This storyboard script fully records the selected target storyboard identifiers and their uniquely corresponding splicing order. The entire process is handled by an intelligent agent, managing tool scheduling and task flow, eliminating the need for manual arrangement of the storyboard order.

[0074] As another example, multiple types of pre-built editing templates are constructed and stored in advance. Different editing templates correspond to different video editing needs in terms of content structure, shot combination logic and clip arrangement paradigm. The templates are pre-configured with corresponding shot selection preferences and general splicing and sorting logic.

[0075] After obtaining the current video editing requirements and storyboard set, the intelligent agent prioritizes matching the appropriate target editing template from the preset editing template library based on the type of editing requirements, usage scenario, and content requirements.

[0076] The intelligent agent independently makes tool invocation decisions, wakes up the script generation tool and sends out the target arrangement template and the full set of storyboard data; the script generation tool relies on the built-in rules of the selected target arrangement template to quickly match and filter target storyboards that fit the template preferences in batches from the storyboard set, directly uses the template's established arrangement logic to lock the splicing order of each target storyboard, quickly solidifies the segment combination relationship, and generates a neat storyboard script with one click.

[0077] In some optional implementations of this embodiment, the above-mentioned execution subject performs step 202 above in the following manner to generate a storyboard script: by calling a script generation tool through an intelligent agent, a storyboard script that meets the video editing requirements is generated based on the description data of the storyboards in the storyboard set.

[0078] Descriptive data is structured information used to characterize the content, external attributes, and content features of a single storyboard shot. It can objectively reflect the inherent characteristics and content attributes of the storyboard itself, providing a basis for storyboard selection and sequential arrangement. For example, it includes independent feature data such as storyboard duration, scene type, camera angle, main content of the scene, overall color tone, camera movement, and content theme tags.

[0079] The agent acquires descriptive data corresponding to each scene within the set. This descriptive data independently carries the content and attribute characteristics of each scene. The agent identifies the predetermined video editing requirements, autonomously calls the script generation tool, and inputs all the scene description data along with the video editing requirements into the script generation tool.

[0080] The script generation tool uses the descriptive data of each storyboard as the core reference, compares it with the constraints of editing requirements, selects suitable target storyboards, and combines the content logic relationship reflected in the descriptive data to reasonably arrange the order of each target storyboard, and finally automatically generates a storyboard script that matches the editing requirements and is logically coherent.

[0081] In this implementation, the descriptive data inherent in the storyboard is used as the basis for arrangement, so that the selection and sequential arrangement of storyboards fit the actual needs of editing, improving the rationality and matching degree of the generated storyboard script, and ensuring the coherence and orderliness of the edited content.

[0082] In some optional implementations of this embodiment, the description data includes attribute description data and semantic description data. Attribute description data is objective and quantifiable underlying physical and audiovisual attribute information that characterizes the storyboard video itself, belonging to objective structured parameters without comprehension bias. Examples include storyboard duration, screen resolution, screen ratio, frame rate, color parameters, screen brightness, camera movement type, screen sharpness, and file format.

[0083] Semantic description data consists of high-level semantic information that characterizes the content, meaning, main information, and theme of a storyboard, reflecting the content connotation and scene logic of the video footage. This includes textual semantic information such as scene type, main subject, actions, environment, theme, narrative meaning, and purpose of the scene.

[0084] In this implementation, before generating the storyboard script, the aforementioned execution entity can also perform the following operations: determine the attribute description data of the storyboards in the storyboard set using a video analysis tool; and determine the semantic description data of the storyboards in the storyboard set using a large artificial intelligence model.

[0085] As an example, for each segment within the storyboard set, an independently deployed video analysis tool, such as FFmpeg (an open-source, cross-platform audio and video processing toolkit), is invoked to detect the underlying audiovisual parameters and basic image attributes of the segment, automatically collect and record various objective indicators, and generate attribute description data corresponding to each segment.

[0086] At the same time, the system calls upon a large artificial intelligence model to perform semantic understanding and content analysis on the content of each scene, identify the scene environment, main subject and content meaning of the scene, and output corresponding semantic description data.

[0087] By associating and binding the attribute description data and semantic description data of the same storyboard, complete storyboard description data is formed, providing a two-dimensional reference for subsequent script generation.

[0088] In this implementation, the storyboard description data is improved from both objective attributes and high-level semantics. By relying on different tools to collect information, the data dimensions are comprehensive and rich, which can effectively improve the accuracy of storyboard selection and script arrangement.

[0089] Step 203: The intelligent agent calls the video splicing tool to splice multiple target scenes according to the storyboard script and generate the edited video.

[0090] In this embodiment, the aforementioned execution entity invokes a video splicing tool through an intelligent agent to splice multiple target storyboards according to the storyboard script, generating an edited video.

[0091] The video splicing tool is an independent functional module with the ability to integrate video clips, connect time sequences, and standardize formats. It can receive calling instructions, storyboards, and corresponding target storyboard materials issued by the intelligent agent. According to the splicing order specified by the storyboard, it can seamlessly integrate multiple independent target storyboards into a complete video file. At the same time, it can optimize the connection of clips and standardize the output format. Its core function is to realize the orderly splicing of storyboard clips and the generation of finished products.

[0092] The video splicing tool can specifically be: a video clip sequential synthesis module, a multi-segment seamless splicing tool, and a standardized video integration component; for example, a component that supports compatible splicing of multi-format storyboard clips, a synthesis tool with basic clip connection and transition optimization, and an integration module that can automatically adapt to the output resolution can all be used as video splicing tools in this method. They can accurately respond to the intelligent agent's call instructions, complete the orderly splicing of the target storyboard according to the storyboard script, and output a complete edited video.

[0093] As an example, firstly, the agent parses the generated storyboard script, extracts all target storyboard identifiers and the clear splicing order recorded in the script, and associates the original material data of the corresponding target storyboard in the storyboard set to complete the material preparation and order verification before splicing, ensuring that the target storyboard identifiers and splicing order are completely consistent with the storyboard script, and avoiding problems such as missing storyboards or disordered order.

[0094] Subsequently, the intelligent agent autonomously initiates a tool call command, accurately calling the video splicing tool and synchronously transmitting the storyboard script and the original material data of the target storyboard to the video splicing tool, clearly informing the tool to "complete the seamless splicing of all target storyboards in the order specified by the storyboard script", without the need to add any additional transition effects or adjust the picture.

[0095] After receiving instructions and relevant data, the video splicing tool first performs format compatibility checks on the target storyboard footage (ensuring only that the footage can be spliced ​​normally, without any other filtering or modification). Then, strictly following the sequence recorded in the storyboard script, it sequentially connects multiple target storyboards to eliminate screen stuttering and frame loss issues at the scene transitions, achieving a seamless transition between storyboards. After splicing is completed, the tool automatically organizes the integrated video file according to a preset standard format (a common format adapted for short video distribution) to generate a complete and coherent edited video.

[0096] Finally, the video stitching tool sends a signal to the agent indicating that the stitching is complete. The agent then confirms that the edited video has been generated, thus completing the video stitching task.

[0097] As another example, after the agent completes the parsing of the storyboard script, it not only extracts the target storyboard identifier and splicing order, but also simultaneously parses the adaptation requirements of the dissemination scenario implied in the video editing requirements (such as the resolution, aspect ratio, and smoothness requirements of short video platforms). Based on these adaptation requirements, it sends a call command with optimization parameters to the video splicing tool to clarify the splicing order, adaptation standards, and optimization direction.

[0098] Then, the intelligent agent calls the video splicing tool and transmits the storyboard script, target storyboard material, and scene adaptation optimization parameters (such as resolution adjustment parameters, transition parameters, etc.) to the tool. After receiving the data, the video splicing tool first performs adaptation preprocessing on each target storyboard, and adjusts the resolution and aspect ratio of the storyboard according to the optimization parameters to ensure that all target storyboards are of uniform specifications and adapted to the preset dissemination scenario.

[0099] Then, the video splicing tool strictly follows the splicing order specified by the storyboard script to splice the pre-processed target storyboards in sequence. At the same time, according to the optimization requirements issued by the intelligent agent, lightweight transition effects (such as fade-in and fade-out, frame connection optimization) are added at the junctions of adjacent storyboards to eliminate the abruptness of storyboard transitions and improve the overall smoothness of the video. For issues such as inconsistent brightness and color tone that occur during the splicing process, slight calibration is automatically performed to ensure that the finished video has a uniform picture.

[0100] Finally, the video splicing tool generates an edited video that meets the scene adaptation requirements according to the preset output format, and simultaneously feeds back the splicing results and optimization status to the agent; the agent confirms that the edited video meets the adaptation requirements and the splicing order is correct, and completes the splicing task.

[0101] In some optional implementations of this embodiment, the execution subject may also perform the following operation: call a video verification tool through an intelligent agent to verify whether the edited video meets the video editing requirements.

[0102] Video verification tools are a collection of functional components that can be scheduled and invoked by an intelligent agent. They are used to automatically detect, compare, and verify the compliance of edited videos from multiple dimensions based on preset video editing requirements, and output quantitative verification results. Examples include duration verification tools, scene continuity detection tools, segment sequence compliance verification tools, and basic image quality detection tools; among them, the duration verification tool is used to verify whether the total duration of the final video and the duration of individual shots meet the required standards.

[0103] As an example, after the edited video is generated, the agent triggers a post-verification process, autonomously invoking a video verification tool. The agent simultaneously inputs the edited video file and pre-determined video editing requirements into the video verification tool. The video verification tool breaks down the various constraints in the editing requirements and performs corresponding tests in sequence, covering aspects such as duration compliance, segment splicing order, and frame continuity. The tool compares the test results with the requirement standards item by item, comprehensively determines whether the overall finished product meets the preset editing requirements, forms a standardized verification conclusion, and sends it back to the agent.

[0104] In this implementation, a dedicated video verification tool is scheduled by an intelligent agent to automatically verify the degree of matching between the finished product and the editing requirements, and to identify compliance deviations in a timely manner, thereby improving the quality controllability and overall compliance rate of the video editing product.

[0105] In some optional implementations of this embodiment, the execution subject may also perform the following operations: in response to the edited video not meeting the video editing requirements, the intelligent agent calls the script generation tool to generate a new storyboard script, and the intelligent agent calls the video splicing tool to generate a new edited video based on the new storyboard script.

[0106] When the editing process is executed for the first time, the agent calls the script generation tool to generate an initial storyboard script based solely on the storyboard set, storyboard description data, and initial video editing requirements. Then, it calls the video splicing tool to complete the first editing. In this implementation, after the agent triggers the correction process, it first analyzes the specific non-compliance items reported by the video verification tool (such as disordered storyboard order, deviation in target storyboard selection, and inconsistent style of the final film), and transforms these non-compliance items into explicit script adjustment constraints.

[0107] Subsequently, the agent re-invoked the script generation tool. Compared to the first invoice, this invoice additionally synchronized the adjustment constraints corresponding to the non-compliant items. The script generation tool was instructed to avoid the non-compliant issues in the first script based on the original set of storyboards and storyboard description data, re-select the appropriate target storyboards, adjust the storyboard splicing order, and generate a new storyboard script that meets the correction requirements, ensuring that the new script specifically addresses the non-compliant issues of the first edit.

[0108] After the new storyboard is generated, the agent calls the video splicing tool again. Unlike the first splicing, this time the new storyboard and splicing optimization prompts for non-compliant items are transmitted simultaneously (such as adjusting the smoothness of the storyboard transitions and ensuring that the target storyboard is selected completely). The video splicing tool is instructed to strictly follow the new storyboard to splice the target storyboard and generate a new edited video.

[0109] If the newly generated edited video still does not meet the video editing requirements after being verified again by the video verification tool, the above correction process is repeated until an edited video that meets the video editing requirements is obtained before reaching the preset repetition threshold, or if an edited video that meets the video editing requirements is still not obtained even after reaching the preset repetition threshold: the agent re-analyzes new non-compliant items, calls the script generation tool to generate an updated storyboard script, calls the video splicing tool to generate a new edited video, until a finished product that meets the video editing requirements is generated. During the repeated execution, the script and splicing parameters are optimized based on the non-compliant feedback of the previous round each time, and the deviation is gradually corrected, which is clearly distinguished from the first execution without feedback and without optimization logic.

[0110] In this implementation, non-compliance issues in the edited video are addressed through automated correction and iteration. New scripts are generated and re-stitched using intelligent agent linkage tools, avoiding manual intervention, improving the efficiency of editing to meet standards, and ensuring that the final product meets the preset editing requirements.

[0111] In some optional implementations of this embodiment, the above-mentioned execution entity may also perform the following operations: determine the frame semantic data of key frames in the edited video; establish the association between the edited video and the frame semantic data.

[0112] Keyframes are representative video frames in edited videos that accurately represent the core content, main information, or key scenes of the video. They are iconic frames that reflect the core content of the video and can serve as the core carrier for video content retrieval and semantic association.

[0113] Frame semantic data refers to the structured and understandable semantic information formed after semantic analysis of the content, main objects, scene meaning, actions, and main themes of keyframes in an edited video. This information is used to accurately describe the content connotation of the keyframes. Examples include the main subject of the keyframe (e.g., "front view of a mobile phone product" or "person's explanation action"), scene type (e.g., "indoor product demonstration scene" or "outdoor science popularization scene"), core actions (e.g., "product unboxing" or "button operation"), and main themes (e.g., "product function introduction" or "knowledge point explanation").

[0114] As an example, after the edited video is generated and verified to meet the editing requirements, the video post-processing stage begins. First, keyframe extraction is performed. A video keyframe extraction tool is invoked to automatically identify and extract keyframes based on the content logic, frame changes, and main features of the edited video. Frames with clear main subjects, representative content, and those reflecting the core segments of the video are prioritized, such as introductory frames, core content display frames, and concluding summary frames. Simultaneously, blurry, redundant, or unrepresentative non-keyframes are removed, forming a keyframe set that ensures the keyframes completely cover the core content of the edited video.

[0115] Next, frame semantic data is acquired. The large-scale artificial intelligence model is invoked, and the extracted keyframe set is input into the model one by one. The model performs deep semantic analysis on each keyframe, identifying core information such as the main subject, scene environment, actions, and content meaning within the keyframe. This semantic information is then structured and organized to generate frame semantic data corresponding to each keyframe, ensuring that the frame semantic data accurately and comprehensively describes the content connotation of the corresponding keyframe without semantic bias.

[0116] Finally, establish the association. A standardized association mapping mechanism is constructed, binding the unique identifier of the edited video (such as the video file identifier) ​​with the identifier of each keyframe and its corresponding semantic data, forming a one-to-one correspondence of "edited video—keyframe—frame semantic data". Simultaneously, this association is stored in a designated database, and an association index is established to ensure that the corresponding keyframe and frame semantic data can be quickly retrieved using the video identifier, or that the corresponding edited video can be located using the frame semantic data.

[0117] In this implementation, by extracting keyframes and generating corresponding frame semantic data, the association between video and semantic information is established, realizing the semantic representation of video content, providing accurate semantic support for subsequent video retrieval and content reuse, and improving the practicality of video post-processing.

[0118] See also Figure 3 , Figure 3 This is a schematic diagram of an application scenario 300 of the agent-based video editing method according to this embodiment. User 301 sends a video to be edited to server 303 via terminal device 302. Server 303 splits the video to be edited into segments 3032 using a progressive multi-segmentation algorithm 3031, obtaining a segment set; agent 3033 calls script generation tool 3034 to generate a segment script that meets the video editing requirements, wherein the segment script represents the splicing order of multiple target segments in the segment set that meet the video editing requirements; agent 3033 calls video splicing tool 3035 to splice multiple target segments according to the segment script, generating the edited video.

[0119] In this embodiment, a progressive multi-scene splitting algorithm is used to split the scenes in the video to be edited, resulting in a scene set. An agent calls a script generation tool to generate a scene script that meets the video editing requirements. The scene script represents the splicing order of multiple target scenes selected from the scene set. The agent calls a video splicing tool to splice multiple target scenes according to the scene script, generating the edited video. This provides an automated video editing method. The progressive multi-scene splitting algorithm improves the accuracy of scene splitting, and the combination of an agent and related tools to generate scene scripts and execute video editing improves video editing efficiency and the coherence of the edited video.

[0120] Continue to refer to Figure 4 The illustration shows a schematic flow 400 of yet another embodiment of the agent-based video editing method according to the present disclosure. Flow 400 includes the following steps: Step 401: The video to be edited is split using the first preset splitting algorithm to obtain multiple preliminary storyboards.

[0121] Step 402: Extract the spatial features of the video frames in the preliminary storyboard using a two-dimensional convolutional network to obtain a spatial feature sequence.

[0122] Step 403: Extract the temporal features of the spatial feature sequence through a one-dimensional temporal convolutional network.

[0123] Step 404: Combining spatial features, temporal features, and similarity features, the transition center point is determined by the first prediction head, and the gradient transition interval is determined by the second prediction head. The gradient transition interval represents a continuous frame sequence that smoothly transitions between adjacent shots.

[0124] Step 405: Verify the transition center point based on the gradient transition interval, and in response to the verification passing, split the initial storyboard with the transition center point as the split boundary to obtain the storyboard set.

[0125] Step 406: Use a large artificial intelligence model to determine whether a scene in the scene set is a single scene.

[0126] Step 407: In response to "OK or no", delete the storyboard that is not a single storyboard from the storyboard set.

[0127] Step 408: Filter out scenes in the scene set that do not conform to the preset filtering rules.

[0128] Step 409: The intelligent agent calls the script generation tool to generate a storyboard script that meets the video editing requirements based on the description data of the storyboards in the storyboard set.

[0129] Step 410: The intelligent agent calls the video splicing tool to splice multiple target scenes according to the storyboard script and generate the edited video.

[0130] Step 411: The intelligent agent calls the video verification tool to verify whether the edited video meets the video editing requirements.

[0131] Step 412: In response to the fact that the edited video does not meet the video editing requirements, the agent calls the script generation tool to generate a new storyboard script, and the agent calls the video splicing tool to generate a new edited video based on the new storyboard script.

[0132] The process 400 of the agent-based video editing method in this embodiment, compared with the above-mentioned process 200, specifically describes a progressive segmentation process and an iterative video editing process, which further improves the efficiency of video editing and the coherence of the edited video.

[0133] Continue to refer to Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a video editing device based on an intelligent agent. This system embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0134] like Figure 5 As shown, the agent-based video editing device 500 includes: a storyboard splitting unit 501, configured to split the storyboards in the video to be edited using a progressive multi-storyboard splitting algorithm to obtain a storyboard set; a script generation unit 502, configured to generate a storyboard script that meets the video editing requirements by calling a script generation tool through an agent, wherein the storyboard script represents the splicing order of multiple target storyboards that meet the video editing requirements in the storyboard set; and a video editing unit 503, configured to splice multiple target storyboards according to the storyboard script by calling a video splicing tool through an agent to generate an edited video.

[0135] In some optional implementations of this embodiment, the storyboard splitting unit 501 is further configured to: split the video to be edited using a first preset splitting algorithm to obtain multiple preliminary storyboards; and split the multiple preliminary storyboards using a second preset splitting algorithm to obtain a storyboard set.

[0136] In some optional implementations of this embodiment, the above apparatus further includes: a storyboard verification unit (not shown in the figure), configured to: determine whether a storyboard in the storyboard set is a single storyboard through an artificial intelligence big data model; and delete storyboards in the storyboard set that are not single storyboards in response to the determination that no.

[0137] In some optional implementations of this embodiment, the above-mentioned device further includes: a storyboard filtering unit (not shown in the figure), configured to: filter storyboards in the storyboard set that do not conform to preset filtering rules.

[0138] In some optional implementations of this embodiment, the segmentation unit 501 is further configured to: extract spatial features of video frames in the preliminary segmentation through a two-dimensional convolutional network to obtain a spatial feature sequence; extract temporal features of the spatial feature sequence through a one-dimensional temporal convolutional network; combine spatial features, temporal features and similarity features between adjacent video frames in the preliminary segmentation to determine the transition center point in the preliminary segmentation; and split the preliminary segmentation with the transition center point as the segmentation boundary.

[0139] In some optional implementations of this embodiment, the scene splitting unit 501 is further configured to: combine spatial features, temporal features and similarity features to determine the transition center point through the first prediction head, and determine the gradient transition interval through the second prediction head, wherein the gradient transition interval represents a continuous frame sequence of smooth transitions between adjacent scenes; verify the transition center point according to the gradient transition interval; and in response to the verification being successful, split the initial scene with the transition center point as the splitting boundary.

[0140] In some optional implementations of this embodiment, the script generation unit 502 is further configured to: call the script generation tool through an agent to generate a storyboard script that meets the video editing requirements based on the description data of the storyboards in the storyboard set.

[0141] In some optional implementations of this embodiment, the description data includes attribute description data and semantic description data, and the above-mentioned apparatus further includes: a data determination unit (not shown in the figure), configured to: determine the attribute description data of the scenes in the scene set through a video analysis tool; and determine the semantic description data of the scenes in the scene set through an artificial intelligence big data model.

[0142] In some optional implementations of this embodiment, the above-mentioned device further includes: a video verification unit (not shown in the figure), configured to: call a video verification tool through an intelligent agent to verify whether the edited video meets the video editing requirements.

[0143] In some optional implementations of this embodiment, the above-mentioned device further includes: a video update unit (not shown in the figure), configured to: in response to the edited video not meeting the video editing requirements, call a script generation tool through an intelligent agent to generate a new storyboard script, and call a video splicing tool through an intelligent agent to generate a new edited video based on the new storyboard script.

[0144] In some optional implementations of this embodiment, the above apparatus further includes: a post-processing unit (not shown in the figure), configured to: determine the frame semantic data of key frames in the edited video; and establish the association between the edited video and the frame semantic data.

[0145] This embodiment provides a video editing device based on an intelligent agent. A segmentation unit uses a progressive multi-segmentation algorithm to segment the segments in the video to be edited, obtaining a segment set. A script generation unit uses the intelligent agent to call a script generation tool to generate a segment script that meets the video editing requirements. The segment script represents the splicing order of multiple target segments in the segment set that meet the video editing requirements. A video editing unit uses the intelligent agent to call a video splicing tool to splice multiple target segments according to the segment script, generating the edited video. This provides an automated video editing method. The progressive multi-segmentation algorithm improves segment accuracy, and the combination of the intelligent agent and related tools to generate segment scripts and execute video editing improves video editing efficiency and the coherence of the edited video.

[0146] According to embodiments of this disclosure, this disclosure also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the agent-based video editing method described in any of the above embodiments.

[0147] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the agent-based video editing method described in any of the above embodiments when executed.

[0148] This disclosure provides a computer program product that, when executed by a processor, can implement the agent-based video editing method described in any of the above embodiments.

[0149] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0150] like Figure 6As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded into random access memory (RAM) 603 from storage unit 608. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0151] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0152] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as an agent-based video editing method. For example, in some embodiments, the agent-based video editing method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the agent-based video editing method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the agent-based video editing method by any other suitable means (e.g., by means of firmware).

[0153] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0154] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable agent-based video editing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0155] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0156] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0157] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0158] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are hosting products within the cloud computing service system to address the management difficulties and weak business scalability inherent in traditional physical hosts and Virtual Private Servers (VPS) services; they can also be servers for distributed systems or servers incorporating blockchain technology.

[0159] According to the technical solution of the embodiments of this disclosure, a video editing method and apparatus based on an intelligent agent are provided. The method involves splitting the video to be edited into a set of scenes using a progressive multi-scene splitting algorithm; generating a scene script that meets the video editing requirements by calling a script generation tool through an intelligent agent, wherein the scene script represents the splicing order of multiple target scenes selected from the scene set; and then, the intelligent agent calls a video splicing tool to splice multiple target scenes according to the scene script, generating the edited video. This provides an automated video editing method. The progressive multi-scene splitting algorithm improves the accuracy of scene splitting, and the combination of an intelligent agent and related tools to generate scene scripts and execute video editing improves video editing efficiency and the coherence of the edited video.

[0160] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0161] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A video editing method based on intelligent agents, comprising: A progressive multi-scene splitting algorithm is used to split the scenes in the video to be edited, resulting in a set of scenes; The intelligent agent calls the script generation tool to generate a storyboard script that meets the video editing requirements. The storyboard script represents the splicing order of multiple target storyboards in the storyboard set that meet the video editing requirements. The intelligent agent invokes a video splicing tool to splice multiple target scenes according to the scene script, generating an edited video.

2. The method according to claim 1, wherein, The progressive multi-scene splitting algorithm for splitting scenes in the video to be edited includes: The video to be edited is split using a first preset splitting algorithm to obtain multiple preliminary storyboards; The second preset splitting algorithm is used to split the initial storyboard into multiple storyboards to obtain the storyboard set.

3. The method according to claim 2, wherein, Also includes: The artificial intelligence model is used to determine whether a scene in the scene set is a single scene. In response to a no-determined condition, delete any storyboard in the storyboard set that is not a single storyboard.

4. The method according to claim 3, wherein, Also includes: Filter out storyboards in the storyboard set that do not conform to the preset filtering rules.

5. The method according to claim 2, wherein, The process of splitting multiple preliminary storyboards using a second preset splitting algorithm includes: Spatial features of video frames in the preliminary storyboard are extracted using a two-dimensional convolutional network to obtain a spatial feature sequence; Temporal features of the spatial feature sequence are extracted using a one-dimensional temporal convolutional network. By combining the spatial features, the temporal features, and the similarity features between adjacent video frames in the preliminary shot, the transition center point in the preliminary shot is determined; The initial storyboard is split using the transition center point as the split boundary.

6. The method according to claim 5, wherein, The step of determining the transition center point in the preliminary shot by combining the spatial features, the temporal features, and the similarity features between adjacent video frames in the preliminary shot includes: Combining the spatial features, temporal features, and similarity features, the transition center point is determined by a first prediction head, and the gradient transition interval is determined by a second prediction head. The gradient transition interval represents a continuous frame sequence with a smooth transition between adjacent shots; and The step of splitting the initial sequence using the transition center point as the split boundary includes: Verify the transition center point based on the described gradient transition range; In response to successful verification, the initial storyboard is split using the transition center point as the split boundary.

7. The method according to any one of claims 1-6, wherein, The process of generating a storyboard script that meets the needs of video editing by calling a script generation tool through an intelligent agent includes: The intelligent agent invokes the script generation tool to generate a storyboard script that meets the video editing requirements, based on the description data of the storyboards in the storyboard set.

8. The method according to claim 7, wherein, The description data includes attribute description data and semantic description data, and Also includes: The attribute description data of the scenes in the scene set are determined by video analysis tools; The semantic description data of the storyboards in the storyboard set is determined by using a large artificial intelligence model.

9. The method according to any one of claims 1-6, wherein, Also includes: The intelligent agent invokes a video verification tool to verify whether the edited video meets the video editing requirements.

10. The method according to any one of claims 1-6, wherein, Also includes: In response to the edited video not meeting the video editing requirements, the agent invokes the script generation tool to generate a new storyboard script, and then invokes the video splicing tool to generate a new edited video based on the new storyboard script.

11. The method according to any one of claims 1-10, wherein, Also includes: Determine the frame semantic data of keyframes in the edited video; Establish the association between the edited video and the semantic data of the frames.

12. A video editing device based on intelligent agents, comprising: The scene splitting unit is configured to use a progressive set of multiple scene splitting algorithms to split the scenes in the video to be edited, resulting in a set of scenes; The script generation unit is configured to call a script generation tool through an intelligent agent to generate a storyboard script that meets the video editing requirements. The storyboard script represents the splicing order of multiple target storyboards in the storyboard set that meet the video editing requirements. The video editing unit is configured to invoke a video splicing tool through the intelligent agent, splice multiple target scenes according to the scene script, and generate an edited video.

13. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.

15. A computer program product comprising: A computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

Images