Short play video editing method, system and device based on multiple modes and medium

Abstract

The invention discloses a multi-mode-based short episode video editing method, system and device and a medium, and the method comprises the steps: carrying out the preprocessing of an original short episode video, and obtaining a second short episode video, subtitles and a subtitle timestamp; analyzing the subtitles, and generating a short episode abstract according to an analysis result of the subtitles; according to the short episode abstract and the highlight editing prompt, performing plot analysis and editing on the second short episode video, and performing editing to form a first video set; scoring videos in the first video set according to a multi-dimensional scoring rule, and screening out M highlight video clips before scoring; and adjusting the timestamps of the M highlight video clips according to the subtitle timestamps, and outputting the adjusted M highlight video clips. According to the method and the device, the full link from short video input to short play mixed video output is constructed, a user only needs to input the to-be-edited short play video, the mixed video can be automatically generated, and the video editing efficiency is improved while the video editing threshold is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of video processing technology, and in particular relates to a multimodal short play video editing method, system, device and medium. Background Art

[0002] With the development of internet technology and the popularity of mobile devices, short video platforms such as Douyin and Kuaishou have rapidly emerged. As an emerging content format, short dramas, with their compact plots, vivid characters, and fast pace, have attracted a large audience. However, with the explosive growth in the number of short dramas, creators face the challenge of efficiently selecting key plot points from the vast amount of video footage and accurately stitching and processing them with special effects.

[0003] Currently, users typically rely on manual editing or semi-automated tools to filter key plot points from massive amounts of video footage and perform splicing and special effects processing. However, with the explosive growth of short video content, the efficiency and intelligence of traditional methods are no longer able to meet market demand. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a short drama video editing method, system, device and medium based on multimodality to solve the problem in related technologies that short drama video editing is not intelligent enough and has low efficiency.

[0005] In order to solve the above technical problems, this application provides the following technical solutions:

[0006] In a first aspect, the present application provides a multimodal short play video editing method, comprising:

[0007] Obtaining an original short play video, and preprocessing the original short play video to obtain a second short play video, subtitle data, and a timestamp corresponding to the subtitles;

[0008] Identifying and parsing the subtitle data, and generating a skit summary based on the subtitle parsing result;

[0009] Analyze and edit the plot of the second short play video according to the short play summary and preset highlight editing prompts, generate N highlight video clips through editing, and form the N highlight video clips into a first video set; where N is a positive integer greater than 0;

[0010] Scoring the highlight video clips in the first video set according to a preset multi-dimensional scoring rule, and selecting the top M highlight video clips based on the scores, where M is a positive integer greater than 0;

[0011] The timestamps of the M highlight video segments are adjusted according to the timestamps corresponding to the subtitle data, and the adjusted M highlight video segments are output.

[0012] Furthermore, the pre-processing of the original short play video to obtain the second short play video, subtitle data, and timestamps corresponding to the subtitles includes:

[0013] Cut off the opening and ending credits of each episode of the short play video, and synthesize the cut-off episodes of each short play to form a second short play video;

[0014] A subtitle extraction tool is used to extract the subtitles in each episode of the short play to obtain the subtitle data in the short play and the timestamp corresponding to the subtitle data.

[0015] Furthermore, the subtitle data is identified and parsed, and a summary of the short play content is generated based on the result of the subtitle parsing, including:

[0016] Use a large language model to identify and parse subtitle data and generate subtitle parsing results;

[0017] A summary of the skit content is generated based on the result of the subtitle parsing.

[0018] Furthermore, the content summary of the short play includes: the core plot, the relationship between the characters and the emotional context.

[0019] Furthermore, the preset highlight editing prompts include: conflict outbreak points, suspense establishment points, visual surprise points and plot reversal points in the video.

[0020] Furthermore, the preset multi-dimensional scoring rules include: content attractiveness dimension, editing rhythm and fluency dimension, emotion and atmosphere communication dimension, information integrity and comprehensibility dimension, and guidance conversion ability dimension.

[0021] Furthermore, adjusting the timestamps of the M highlight video segments according to the timestamps corresponding to the subtitle data, and outputting the adjusted M highlight video segments, includes:

[0022] Determining, based on timestamps corresponding to the subtitles, whether any subtitle sentences in the M highlight video segments are truncated;

[0023] If so, the timestamp of the highlight video segment is adjusted, and the adjusted M highlight video segments are output.

[0024] In a second aspect, the present application also provides a multimodal short play video editing system, comprising:

[0025] An acquisition module, configured to acquire an original short play video and pre-process the original short play video to obtain a second short play video, subtitle data, and a timestamp corresponding to the subtitles;

[0026] A subtitle parsing module, configured to identify and parse the subtitle data, and generate a skit summary based on the subtitle parsing result;

[0027] a highlight video clip editing module, configured to analyze and edit the plot of the second short play video based on the short play summary and preset highlight editing prompts, generate N highlight video clips through editing, and form the N highlight video clips into a first video set; wherein N is a positive integer greater than 0;

[0028] a scoring and screening module, configured to score the highlight video clips in the first video set according to a preset multi-dimensional scoring rule, and screen out the top M highlight video clips based on the scores, where M is a positive integer greater than 0;

[0029] The adjustment and output module is configured to adjust the timestamps of the M highlight video segments according to the timestamps corresponding to the subtitle data, and output the adjusted M highlight video segments.

[0030] In a third aspect, the present application also provides a computer electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above-described short play video editing methods when executing the computer program.

[0031] In a fourth aspect, the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the steps of any of the above-mentioned short drama video editing methods.

[0032] The present application provides a multimodal short play video editing method, system, device, and medium, which have the following beneficial effects:

[0033] This application builds a complete chain process from short video input to short drama mashup video output. Users only need to input the short drama video to be edited, and the mashup video of the short drama will be automatically generated, which greatly shortens the video production time and reduces the need for professionals. This makes it easy for users without professional backgrounds to produce high-quality video content. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 This is a flowchart of a multimodal short play video editing method according to an embodiment of the present application;

[0036] Figure 2 This is a structural diagram of a short play video editing system based on multimodality according to an embodiment of the present application;

[0037] Figure 3 It is a structural diagram of a computer electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0039] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. Conversely, when an element is referred to as being "directly on" another element, there is no intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only.

[0040] In this application, unless otherwise expressly specified or limited, terms such as "mounted," "connected," "connect," and "fixed" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integration; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components or interactions between two components. Those skilled in the art will understand the specific meanings of these terms in this application based on specific circumstances.

[0041] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0042] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the", and "the" used in one or more embodiments of the present application are also intended to include plural forms unless the context clearly indicates otherwise.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used in the template description herein are for the purpose of describing specific embodiments only and are not intended to limit this application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0044] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when..." or "when...".

[0045] Currently, the existing video editing technologies include the following:

[0046] 1. Rule-based keyframe extraction:

[0047] This method uses a set of preset rules to identify key frames or segments in a video, such as shot switching frequency, brightness changes, and color contrast. While this method can capture some significant changes to a certain extent, it lacks generalizability and struggles to adapt to complex scenarios and diverse content styles. Furthermore, because these rules are typically static and fixed, manual adjustments to the rule set are often required for different types or styles of short videos (e.g., dramatic plot twists versus lighthearted and humorous moments), increasing operational complexity and time costs.

[0048] 2. Single modal feature analysis:

[0049] Unimodal feature analysis usually relies only on a single type of data for segment screening. For example: 1. Visual information processing: Use computer vision technology to identify the actions in the video to determine which parts may contain highlight moments. However, this approach ignores the importance of audio signals. In fact, music, dialogue and other sound elements are often the key factors in determining whether a scene is a "highlight". 2. Text information processing: Use the model to extract subtitles or match keywords after speech-to-text to screen important segments. Although this method can capture certain specific plots or lines, it cannot fully understand the overall emotional atmosphere and non-verbal expressions of the video, which may cause important emotional climaxes to be missed. Therefore, it is difficult to achieve a comprehensive understanding and accurate editing of short video content by relying solely on a single dimension of information for analysis, which limits the quality and appeal of the final work.

[0050] 3. Traditional machine learning model:

[0051] Traditional machine learning models, such as support vector machines (SVM) and random forests, were once the mainstream methods for solving classification problems. However, when applied to the automatic remixing of short drama highlights, these models have obvious limitations: 1. Manually labeled features: These models usually need to rely on a large amount of manually labeled data as training samples, which is not only time-consuming and labor-intensive, but also easily affected by subjective bias. At the same time, due to the huge differences in content between different types of short videos, manually labeled features may not be applicable to all situations, resulting in limited model generalization capabilities. 2. Insufficient ability to process multimodal data: Short video content contains rich multimodal information (images, audio, text, etc.), and traditional machine learning models perform poorly in fusing these different types of data, making it difficult to fully utilize all available information to improve editing accuracy and efficiency.

[0052] The above method has the following disadvantages:

[0053] Multimodal information fragmentation:

[0054] Current methods are generally unable to effectively integrate multimodal features such as vision, audio, and text. For example, some systems may rely solely on motion or color changes in the video to identify potential highlight clips, while ignoring the emotional color of the background music or the importance of the dialogue content. This single-dimensional information processing method leads to a one-sided understanding of the key plot of the skit, reducing the quality of the final remix. In addition, current methods lack synergy. There is an intrinsic connection between different modal data (such as images and sounds), which jointly affect the audience's feelings. However, existing technologies often fail to fully explore the relationship between these data, resulting in the generated remix clips that cannot fully reflect the emotional fluctuations and narrative rhythm of the original skit.

[0055] Lack of intelligence:

[0056] Manual rule-based approaches struggle to flexibly adapt to the diverse styles of skits. For example, creating tension in a suspense film requires distinct editing strategies from delivering humor in a comedy. Because rules are often pre-set and fixed, these approaches struggle with diverse content and are unable to adaptively adjust editing logic to meet the needs of different skit types.

[0057] Inefficiency:

[0058] Traditional machine learning models exhibit high computational complexity when processing massive amounts of material, especially when real-time analysis of large numbers of videos is required. This consumes huge computing resources, resulting in slow processing speeds and making it difficult to meet the requirements of short video platforms for rapid content updates.

[0059] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes in certain embodiments will not be repeated. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0060] Please refer to Figure 1 The embodiment of the present application provides a multimodal short play video editing method, which includes at least the following steps:

[0061] S10: Obtain an original short play video, and pre-process the original short play video to obtain a second short play video, subtitle data, and a timestamp corresponding to the subtitles.

[0062] In one embodiment of the present application, the above step S10 includes:

[0063] S11. Cut off the opening and ending credits of each episode of the short drama video, and synthesize the cut-off episodes of each episode to form a second short drama video.

[0064] S12. Using a subtitle extraction tool, extract the subtitles in each episode of the short play to obtain subtitle data in the short play and a timestamp corresponding to the subtitle data.

[0065] In this embodiment, the complete subtitles and timestamps can be extracted through the Whisper-large-v3 model, and the redundant segments at the beginning and end of each episode can be removed to generate a merged video.

[0066] S20: Identify and parse the subtitle data, and generate a skit summary based on the subtitle parsing result.

[0067] In one embodiment of the present application, the above step S20 includes:

[0068] S21. Use a large language model to identify and parse the subtitle data to generate subtitle parsing results.

[0069] S22. Generate a summary of the skit content based on the subtitle parsing result.

[0070] Specifically, in this embodiment, a large language model (LLM) is first used to identify and parse subtitle data to obtain subtitle parsing results.

[0071] Secondly, after obtaining the results of subtitle parsing, the preset skit introduction prompt (prompt1) is used to guide the gemini-2.0 visual model (VLM) to analyze the content of these videos and generate structured skit summaries.

[0072] Specifically, the preset introduction prompt may include the following:

[0073]

[0074]

[0075] It should be noted that:

[0076] -Even if the subtitles are incomplete, try to reasonably complete the plot.

[0077] - All content should be written in Chinese, using natural, concise and professional language.

[0078] -The output must conform to the JSON format, with a comma after each item and a comma after the last}.

[0079] In one embodiment of the application, the short play summary includes: the core plot, the relationships between the characters, and the emotional context.

[0080] S30. According to the short play summary and preset highlight editing prompts, the second short play video is subjected to plot analysis and editing, N highlight video clips are generated by editing, and the N highlight video clips are formed into a first video set; wherein N is a positive integer greater than 0.

[0081] Specifically, in this example, the most representative and engaging clips are automatically identified and selected as highlight clips by combining the skit summary and the highlight clip prompt (prompt2). This process may involve techniques such as sentiment analysis and keyword matching to ensure that the selected clips are able to capture the audience's attention. Ultimately, a set of timestamps TT is output, indicating which parts should be included in the video mix. In other words, the videos within these timestamps are the highlight videos.

[0082] In one embodiment of the present application, the preset highlight editing prompts include: conflict outbreak points, suspense establishment points, visual surprise points, and plot reversal points in the video.

[0083] Specifically, the highlight blending prompt here is the prompt we gave to the large model gemini-2.0, and the specific content is as follows:

[0084] **Character setting**: You are a professional short video editor who is good at extracting highly contagious "hook" clips from multiple episodes of short dramas.

[0085] Please follow the steps below to analyze my skit video:

[0086] **Step 1-Content Structure Analysis**

[0087] -Analyze the core events of each episode according to the timeline and mark the following key points:

[0088] -Conflict flashpoints (e.g. arguments, confrontations)

[0089] -Suspense building point (such as 3 seconds before the secret is revealed)

[0090] - Emotional highs (such as crying, laughing, anger)

[0091] - Visual spectacles (such as action scenes, special effects shots)

[0092] - plot twists (such as identity revelation, unexpected events)

[0093] **Step 2-Hook Fragment Candidate Pool**

[0094] - Filter the clips that meet the following criteria from the above key points:

[0095] - Duration: 10-15 seconds of continuous footage (non-jump cuts)

[0096] -Information: Contains hints about character relationships / core conflicts but does not spoil the plot.

[0097] -Emotional intensity: sudden changes in background music / dramatic changes in character expressions / impactful dialogue

[0098] -Visual priority: close-up > medium shot > wide shot

[0099] - The selected fragments must be complete and not truncated. For example, the last line of the dialogue should be "I love you" but it is cut off at the word "I".

[0100] **Step 3-Output Requirements**

[0101] -Give the top 3 candidate segments, each with a duration of 10 seconds, and output them in the following format (accurate to milliseconds, without any other text):

[0102] (00:xx-00:xx)

[0103] (00:xx-00:xx)

[0104] (00:xx-00:xx)

[0105] EXCLUSION RULE

[0106] - Do not select clips that contain spoilers of key plot points

[0107] -Skip pure dialogue / daily scenes

[0108] S40. Score the highlight video clips in the first video set according to a preset multi-dimensional scoring rule, and select the top M highlight video clips according to the scores, where M is a positive integer greater than 0.

[0109] Specifically, in this embodiment, each highlight video clip in the first video set is scored according to a multi-dimensional scoring rule. After scoring, the system sorts the candidate videos by score and selects the top M highlight video clips from the first video set based on the scores. For example, the top three highlight video clips are selected.

[0110] It should be noted that the scoring rules can be set based on multiple dimensions such as video quality, plot coherence, and audience interest.

[0111] In one embodiment of the present application, the preset multi-dimensional scoring rules include: content attractiveness dimension, editing rhythm and fluency dimension, emotion and atmosphere communication dimension, information completeness and comprehensibility dimension, and guidance conversion ability dimension.

[0112] Specifically, the multi-dimensional scoring rules are as follows:

[0113]

[0114]

[0115]

[0116] S50 . Adjust the timestamps of the M highlight video segments according to the timestamps corresponding to the subtitle data, and output the adjusted M highlight video segments.

[0117] In a certain embodiment of the present application, the above step S50 includes:

[0118] S51. Determine, based on the timestamps corresponding to the subtitles, whether any subtitle sentences in the M highlight video segments are truncated;

[0119] S52: If yes, adjust the timestamp of the highlight video segment, and output the adjusted M highlight video segments.

[0120] Specifically, in order to avoid sentences being truncated or switching unnaturally when editing the video, in this embodiment, it is necessary to check and judge the above-mentioned highlight video clips. If the above-mentioned situation occurs, the above-mentioned highlight video needs to be adjusted. For example, when the scene in the highlight video is "the male and female protagonists confessing their love", but the entire process is not edited during the video editing process, at this time, the highlight video clip needs to be adjusted so that the highlight video clip includes the confession process.

[0121] It's important to note that this application adjusts videos based on their timestamps. This process leverages the previously generated complete subtitles, Tsubtitle, to adjust timestamps, ensuring smooth transitions between each segment. The resulting highlight remix video, PP, showcases the most exciting parts while maintaining narrative coherence.

[0122] It is understandable that, among the M highlight video clips output, the user can select the best highlight video clip as the short drama highlight editing video according to actual needs.

[0123] In order to better understand this application, a specific example is given below:

[0124] 1. Generate timestamps and subtitles.

[0125] First, the system receives a user-entered collection V of short drama videos. It processes the input video collection V using the Whisper-large-v3 model, which efficiently converts speech into text and assigns accurate timestamps to each conversation. This step not only generates subtitles Tsubtitle for the entire video but also provides the start and end timestamps for each sentence in the video. This is crucial for subsequent extraction of highlight clips, as it allows us to precisely locate specific scenes or conversations.

[0126] Ttimestamp,Tsubtitle=WhisperLargeV3(V)

[0127] Use the Whisper-large-v3 model to generate timestamps and subtitles for the full video.

[0128] 2. Generate a short play introduction

[0129] The system guides the Gemini-2.0 Visual Model (VLM) to analyze the content of these videos based on a preset short play introduction prompt (prompt1). Using Whisper to identify and parse subtitle files (srt), the LLM understands key information such as the plot and characters, and then generates a concise plot summary S. The key to this step is how to effectively extract the core information from the video content and present it concisely for subsequent modules.

[0130] S=GenerateSummary(V,prompt1)

[0131] Among them, the GenerateSummary function guides LLM to identify the srt file based on prompt1 to generate a summary of the short play.

[0132] 3. Extract highlight fragments

[0133] In this embodiment, an extraction agent (named ExtractAgent) can be defined to automatically extract highlight clips. ExtractAgent combines the skit introduction S and the highlight clip prompt (prompt2) to automatically identify and select the most representative and attractive clips as highlights. This process may involve technical means such as sentiment analysis and keyword matching to ensure that the selected clips can attract the audience's attention. Ultimately, ExtractAgent outputs a set of timestamps TT, indicating which parts should be included in the video mix.

[0134] T=ExtractAgent(S,prompt2)

[0135] The ExtractAgent module outputs 5 combinations of highlight clip timestamps through the skit introduction and highlight clip prompt (prompt2).

[0136] 4. Scoring and sorting

[0137] In this embodiment, a scoring agent (named ScoreAgent) can be defined to evaluate and score the candidate videos proposed by ExtractAgent. Scoring criteria can be set based on multiple dimensions such as video quality, plot coherence, and audience interest. After scoring, the system sorts the candidate videos by score and selects the top three videos as the final mashup material Rsorted.

[0138] Rsorted=ScoreAgent(R,prompt3)

[0139] The ScoreAgent module scores the candidate video list and gives the top 3 videos in descending order of scores.

[0140] 5. Correct timestamp

[0141] After determining the top three videos, the system needs to further correct their timestamps to avoid sentence truncation or unnatural transitions. This process uses the previously generated complete subtitles Tsubtitle to adjust the timestamps and ensure smooth transitions between each clip. The resulting highlight remix video P showcases the most exciting parts while maintaining narrative coherence.

[0142] P=CorrectTimestamp(Rsorted,Tsubtitle)

[0143] Use the generated srt subtitles to correct the timestamps of the Top 3 videos to avoid sentence truncation and other situations.

[0144] 6. Output results

[0145] Finally, the optimized highlight mix-and-cut video P is output to the user for review and selection. The user can choose the most satisfactory version according to their preferences.

[0146] Output(P)

[0147] Finally, 3 optimized highlight mixed-cut videos are output for users to choose from.

[0148] The present application provides a multimodal short play video editing method, system, device, and medium, which have the following beneficial effects:

[0149] This application builds a complete chain process from short video input to short drama mashup video output. Users only need to input the short drama video to be edited, and the mashup video of the short drama will be automatically generated, which greatly shortens the video production time and reduces the need for professionals. This makes it easy for users without professional backgrounds to produce high-quality video content.

[0150] See also Figure 2 The embodiment of the present application also provides a multimodal short play video editing system 200, comprising:

[0151] An acquisition module 201 is configured to acquire an original skit video and pre-process the original skit video to obtain a second skit video, subtitle data, and a timestamp corresponding to the subtitles;

[0152] A subtitle parsing module 202 is used to identify and parse the subtitle data and generate a skit summary based on the subtitle parsing result;

[0153] A highlight video clip editing module 203 is configured to analyze and edit the second short play video according to the short play summary and preset highlight editing prompts, generate N highlight video clips through editing, and form a first video set with the N highlight video clips; wherein N is a positive integer greater than 0;

[0154] The scoring and screening module 204 is configured to score the highlight video clips in the first video set according to a preset multi-dimensional scoring rule, and screen out the top M highlight video clips based on the scores, where M is a positive integer greater than 0;

[0155] The adjustment and output module 205 is configured to adjust the timestamps of the M highlight video segments according to the timestamps corresponding to the subtitle data, and output the adjusted M highlight video segments.

[0156] See also Figure 3 The embodiment of the present application also provides a computer electronic device 300, including a memory 303 and a processor 302, wherein the memory 303 stores a computer program, and when the processor executes the computer program, the steps of the short drama video editing method described in any one of the above items are implemented.

[0157] Specifically, the electronic device 300 includes: a transceiver 301, a bus interface and a processor 302, the processor 302 is used to obtain an original short play video, and pre-process the original short play video to obtain a second short play video, subtitle data and a timestamp corresponding to the subtitles; identify and parse the subtitle data, and generate a short play summary based on the result of the subtitle parsing; analyze and edit the plot of the second short play video according to the short play summary and preset highlight editing prompts, edit to generate N highlight video clips, and form the N highlight video clips into a first video set; wherein N is a positive integer greater than 0; score the highlight video clips in the first video set according to a preset multi-dimensional scoring rule, and filter out the top M highlight video clips according to the scores, wherein M is a positive integer greater than 0; adjust the timestamps of the M highlight video clips according to the timestamps corresponding to the subtitle data, and output the adjusted M highlight video clips.

[0158] In the embodiment of the present application, the electronic device 300 further includes: a memory 303. Figure 3In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits such as one or more processors represented by processor 302 and memory represented by memory 303. The bus architecture may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 301 may be a plurality of components, i.e., a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium. The processor 302 is responsible for managing the bus architecture and general processing, and the memory 303 may store data used by the processor 302 when performing operations.

[0159] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned short play video editing methods are implemented.

[0160] In this embodiment, the computer-readable storage medium may be a non-volatile storage medium or a volatile storage medium. For example, the computer storage medium may include, but is not limited to, a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, among other media capable of storing program code.

[0161] In all examples shown and described herein, any specific values should be interpreted as merely exemplary and not limiting, and thus other examples of the exemplary embodiments may have different values.

[0162] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or flowchart, and the combination of boxes in the structure diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0164] In addition, the functional modules or units in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0165] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a terminal device (which can be a smart phone, personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0166] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A short drama video editing method based on multimodality, characterized in that: include: Obtaining an original short play video, and preprocessing the original short play video to obtain a second short play video, subtitle data, and a timestamp corresponding to the subtitles; Identifying and parsing the subtitle data, and generating a skit summary based on the subtitle parsing result; Analyze and edit the plot of the second short play video according to the short play summary and preset highlight editing prompts, generate N highlight video clips through editing, and form the N highlight video clips into a first video set; where N is a positive integer greater than 0; Scoring the highlight video clips in the first video set according to a preset multi-dimensional scoring rule, and selecting the top M highlight video clips based on the scores, where M is a positive integer greater than 0; The timestamps of the M highlight video segments are adjusted according to the timestamps corresponding to the subtitle data, and the adjusted M highlight video segments are output.

2. The short play video editing method according to claim 1, characterized in that: The pre-processing of the original short play video to obtain the second short play video, subtitle data, and timestamps corresponding to the subtitles includes: Cut off the opening and ending credits of each episode of the short play video, and synthesize the cut-off episodes of each short play to form a second short play video; A subtitle extraction tool is used to extract the subtitles in each episode of the short play to obtain the subtitle data in the short play and the timestamp corresponding to the subtitle data.

3. The short play video editing method according to claim 1, characterized in that: The step of identifying and parsing the subtitle data and generating a summary of the short play content based on the subtitle parsing result includes: Use a large language model to identify and parse subtitle data and generate subtitle parsing results; A summary of the skit content is generated based on the result of the subtitle parsing.

4. The short play video editing method according to claim 3, characterized in that: The short play content summary includes: The core plot, character relationships and emotional context.

5. The short play video editing method according to claim 1, characterized in that: The preset highlight editing prompts include: conflict outbreak points, suspense establishment points, visual surprise points and plot reversal points in the video.

6. The short play video editing method according to claim 1, characterized in that: The preset multi-dimensional scoring rules include: content attractiveness dimension, editing rhythm and fluency dimension, emotion and atmosphere communication dimension, information integrity and comprehensibility dimension, and guidance conversion ability dimension.

7. The short play video editing method according to claim 1, characterized in that: The adjusting the timestamps of the M highlight video segments according to the timestamps corresponding to the subtitle data, and outputting the adjusted M highlight video segments, includes: Determining, based on timestamps corresponding to the subtitles, whether any subtitle sentences in the M highlight video segments are truncated; If so, the timestamp of the highlight video segment is adjusted, and the adjusted M highlight video segments are output.

8. A short drama video editing system based on multimodality, characterized in that: include: An acquisition module, configured to acquire an original short play video and pre-process the original short play video to obtain a second short play video, subtitle data, and a timestamp corresponding to the subtitles; A subtitle parsing module, configured to identify and parse the subtitle data, and generate a skit summary based on the subtitle parsing result; a highlight video clip editing module, configured to analyze and edit the plot of the second short play video based on the short play summary and preset highlight editing prompts, generate N highlight video clips through editing, and form the N highlight video clips into a first video set; wherein N is a positive integer greater than 0; a scoring and screening module, configured to score the highlight video clips in the first video set according to a preset multi-dimensional scoring rule, and screen out the top M highlight video clips based on the scores, where M is a positive integer greater than 0; The adjustment and output module is configured to adjust the timestamps of the M highlight video segments according to the timestamps corresponding to the subtitle data, and output the adjusted M highlight video segments.

9. A computer electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the short play video editing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the short play video editing method according to any one of claims 1 to 7.

Images