A video highlight clip automatic extraction method and system based on multi-signal fusion and feedback self-calibration

Abstract

The application discloses a video highlight segment automatic extraction method and system based on multi-signal fusion and feedback self-calibration, and belongs to the technical field of video content processing. The method completes media analysis on a long video to obtain shot boundaries, and obtains a high-precision unified subtitle script through ASR and OCR double-source alignment; high-light candidates are independently generated from semantics, audio and vision, respectively, audio adopts a two-way deviation identification of a relative set rolling baseline to identify two types of dramatic tension segments, namely, burst and silence; in the fusion link, certainty rules are used to complete double adsorption of cut points, shot boundary rejection conditions are set, and after the adsorption of cut points, a duplicate is executed to be merged, cross-modal homologous candidates are sorted and added, and a highlight list is obtained in combination with structure position prior weighting. The application solves problems such as single-signal recall blind area, cut point cutting distortion, model abnormal output, repeated cutting, small sample self-learning overfitting and version caliber confusion.

Description

Technical Field

[0001] This invention relates to the field of video content processing technology, specifically to a method and system for automatically extracting video highlight segments based on multi-signal fusion and feedback self-calibration. Background Technology

[0002] As short video platforms become the main traffic source for long videos such as movies, TV series, animated shorts, and variety shows, extracting highlight clips from long videos and processing them into short videos has become a high-frequency and essential production task for content creators. Manual processing requires viewing the entire episode, identifying key moments, and repeatedly adjusting cut points, with each episode taking hours to process. Current automated highlight clip generation technologies mainly fall into three categories: the first is a low-level audiovisual feature heuristic approach, relying on volume peaks, camera switching frequency, and image motion intensity to construct a heat curve and select extreme value ranges as highlights; the second is based on external user interaction data, using bullet screen density, comment time distribution, and playback heat curves to locate periods of strong user feedback; the third relies on large language models for semantic understanding, inputting subtitle text into the model and directly outputting the start and end times and reasons for highlight clips. Vertical screen conversion mostly uses center-cropping or blurred background filling methods.

[0003] The existing technology has the following drawbacks: 1. Single signals have inherent blind spots and cannot self-detect these blind spots. Pure semantic subtitle links cannot identify visual highlights without dialogue, and scenes without dialogue such as fights, transformations, and disasters in anime shorts cannot be recalled; pure audio judges highlights based on absolute loudness, which will miss low-volume highlights such as whispered suspense hooks; solutions based on external interactive data such as bullet comments and playback heat are only suitable for videos that have already been launched and accumulated playback data, and cannot support business scenarios such as new film premieres or pre-edited releases.

[0004] 2. Poor cut point processing results; unconditional alignment of shot boundaries introduces new errors. Existing solutions directly use candidate original endpoints to cut segments, which cuts off dialogue; directly aligning with shot boundaries results in misalignment between shot and sentence boundaries, with cut points falling in the middle of sentences. In actual testing, only 2 out of 14 sentence cutting checks passed, and it also causes distortion of segment length, stretching a 4-second effective visual segment to 13 seconds.

[0005] 3. The model output lacks hard constraints, and abnormal results flow directly into the final product library. The length of output segments from large models fluctuates greatly, resulting in extremely short fragments or extremely long segments. Without an interception mechanism, these segments will directly enter the segment library. At the same time, multimodal and large language models are at risk of refusing to answer or reporting errors. The lack of a fallback mechanism will cause the entire chain to fail.

[0006] 4. The multi-signal fusion weights have poor adaptability and are prone to overfitting in small sample scenarios. Manually fixing weights cannot adapt to different subjects; directly using manual feedback for end-to-end training makes the model susceptible to interference from a few random feedbacks in small sample scenarios, and the model performance deteriorates with use.

[0007] 5. Repeated generation of highlight fragments. Multiple signals output candidates will hit the same highlight region. Before the cut point is aligned, there will be a difference of several seconds between the candidate intervals, which will not trigger the deduplication. After the cut point is snapped, multiple candidates collapse into the same time interval, resulting in a large number of identical slices.

[0008] 6. The boundaries between cached and historical ranking versions are blurred, and the results cannot be reproduced. After changes to prompts, fusion rules, and model selection, old intermediate caches and historical highlight rankings continue to be reused, resulting in a mix of old and new data, making it impossible to compare effects and reproduce results. Summary of the Invention

[0009] The purpose of this invention is to provide an automatic method and system for extracting video highlight clips based on multi-signal fusion and feedback self-calibration, in order to overcome the technical defects of existing technologies such as single-signal recall blind spots, poor cut point quality, inability to intercept model anomalies, difficulty in weight adaptation, repeated slice generation, and version non-reproducibility. This invention provides an automatic method and system for extracting video highlight clips based on multi-signal fusion and feedback self-calibration, achieving complementary multi-signal recall, deterministic rule-based cut point calculation, small-sample-friendly feedback self-calibration, and version-controllable automatic generation of highlight clips.

[0010] To achieve the above objectives, the present invention provides the following technical solution: an automatic extraction method for video highlight segments based on multi-signal fusion and feedback self-calibration, comprising the following steps: S10, Media Analysis: Obtain the input long video, extract the audio track, and perform shot segmentation to obtain a shot boundary table; the shot boundary table records the start and end times of each shot for subsequent cut point snapping.

[0011] S20. Text Acquisition and Dual-Source Alignment: Transcribe audio tracks using Automatic Speech Recognition (ASR) to output segmented text with timestamps; Recognize embedded subtitles in the video using Optical Character Recognition (OCR); Use ASR output as a timestamp skeleton to cover segments missed by OCR; OCR output provides text truth values ​​to correct ASR recognition errors, and align and merge to obtain a unified subtitle text.

[0012] S30. Audio Emotional Signal Extraction: Two audio curves, the root mean square (RMS) energy and the onset intensity, are extracted from the audio track and robustly normalized using the 95th quantile. A ten-second window rolling average is used to construct the audio baseline for this episode. The absolute value of the deviation of the audio curve values ​​from the rolling baseline is calculated as the main audio signal. Absolute loudness is retained with only a small weight as a fallback output audio emotional intensity envelope, used for scoring candidate segments. High scores are obtained for both explosive high-decibel scenes and sudden silences, which are considered dramatic tension scenes.

[0013] S40. Semantic Candidate Generation: Input the unified subtitle script into the Large Language Model (LLM) and output candidate highlight intervals, highlight types, reasons, semantic intensity scores, and suggested time points for hook endings; optionally, maintain the series character list and main plot scrolling summary as cross-episode memory, input the model to identify cross-episode foreshadowing; automatically switch to the backup model as a fallback when the main model refuses to answer or reports an error.

[0014] S50, Visual Candidate Generation: Transcodes long videos into simplified videos with low resolution, low frame rate, and no audio. The native input is a multimodal model, and the output is a visual impact candidate range and a visual impact score. A three-level degradation strategy is set: when the native video input fails or times out, the keyframes are extracted in a degradation manner; when the keyframe processing fails, an empty candidate list is returned; the visual link only performs incremental recall, and the failure of the link will not interrupt the main process.

[0015] S60. Fusion and tangent point determination: All steps employ deterministic rules; the model does not participate in tangent point calculation. This includes the following sub-steps: S61. Semantic threshold filtering: Candidates with semantic strength lower than the preset threshold are directly discarded; S62, Duration Hard Constraint: Overly long candidates are preferentially pruned to the hook suggestion point output by the model. If there is no valid hook point, it is pruned to the preset duration limit. S63, Dual Adsorption of Cutting Points: The first step iteratively adsorbs sentence boundaries, with the starting point back to the beginning of the subtitle sentence and the ending point extended to the end of the subtitle sentence; for overlapping subtitle times, the starting point is taken as the minimum start time of all sentences spanning that time point, and the ending point is taken as the maximum end time of all sentences spanning that time point, iterating until no sentence crosses the cutting point; the second step attempts to adsorb shot boundaries, setting rejection conditions: the shot boundary corresponding to the starting point must be earlier than the current starting point, and the shot boundary corresponding to the ending point must be later than the current ending point, and the shot boundary must not cross any sentence, and the cutting point is replaced only if the conditions are met; after adsorption is completed, fragment candidates with a duration less than the lower threshold are directly discarded; S64. Adsorption and deduplication: After adsorption at the tangent point is completed, deduplication is performed based on the IoU cross-union ratio. Candidates with cross-union ratios exceeding the threshold are grouped into the same group, and the candidate with the highest semantic score in each group is retained as the group representative. S65. Cross-modal complementary bonus: When the same group has both semantic source candidates and visual source candidates, the group representative candidate increases the score positively correlated with the visual impact score, without adding new slice entries. S66. Prior scoring of structural position: The candidate segment starting point is assigned a score based on the proportion of the total video duration. The high partition threshold is set to 0.65 to adapt to the true distribution of short drama highlights concentrated in 55%-75% of the positions. S67. Weighted Summation and Sorting: Combining semantic score, audio score, and structural position score, the candidate final score is obtained by weighted summation based on the corresponding weights, and the highlight list is output in descending order of score.

[0016] S70, Final Processing: Cuts video clips based on highlight lists; performs landscape to portrait mode conversion, with options for brand bars, blurred background fill, and intelligent subject cropping; automatically generates title text, and supports splicing multiple highlights to generate compilation trailers.

[0017] S80. Manual review and feedback collection: Output highlight lists and slice results to operations personnel; collect manual operation events, record positive examples for adoption, negative examples for rejection, and weak positive examples for fine-tuning cut points; rewriting copy operations do not participate in weight calculation; record the original score of each candidate multi-channel signal in a synchronous manner for each feedback.

[0018] S90, Self-calibration of fusion weights: S91. Filter valid labeled events and distinguish between positive examples, negative examples, and weak positive examples; S92. Discrimination Calculation: Calculate the difference between the mean scores of each signal in the positive and negative example groups as the discrimination; if the discrimination is negative, truncate it to zero; normalize the discrimination of each signal to obtain the learning weight. S93. Confidence Mixing: Calculate the confidence level α = n / (n+K), where n is the number of valid samples and K is a smoothing constant; linearly mix the learned weights and the human prior weights according to the confidence level α, and then normalize; if there are few samples, the weights are biased towards the human prior; if there are enough samples, the weights are biased towards the learned data weights; if either the positive or negative sample is missing, skip the weight calibration. S94, Main Signal Floor Protection: Sets a lower limit for the weight of the semantic main signal. If the weight after mixing is lower than the lower limit, it is raised to the lower limit; the remaining weight is redistributed proportionally among the other signals. After calibration, the learning weight file is stored on disk. The next time the system runs, it is hot-loaded back to the S67 weighted summation step to form a closed-loop feedback.

[0019] S100, Link Version and Cache Failure Control: Maintain a single highlight recognition link version number; any configuration changes that alter the highlight output results, including prompt words, fusion and adsorption rules, model selection, and point marking methods, will trigger version number increment; after the version number is updated, intermediate analysis cache and highlight lists already stored on disk will automatically become invalid, triggering a rerun; ensure that all lists in the database are output with the same caliber, and that the results are reproducible and comparable.

[0020] Furthermore, the method can be adapted to various long-video to short-video conversion scenarios, including animated short dramas, film and television dramas, variety shows, sports events, live replays, teaching videos, and meeting recordings.

[0021] Furthermore, it is possible to increase the density of bullet comments, facial expressions, and motion vectors as additional signal channels; it is also possible to remove audio signal links for materials without audio tracks.

[0022] Furthermore, the audio rolling baseline can be replaced with median filtering, exponential moving average, and wavelet low-frequency components; the deviation calculation can be replaced with Z-score standard score and relative rate of change; and the normalized quantiles can be adjusted as needed.

[0023] Furthermore, sentence boundaries can be replaced with silence intervals from speech endpoint detection outputs; shot boundaries can be replaced with music beat points; and the rejection logic for boundary snapping remains unchanged.

[0024] Furthermore, the self-calibration module's discrimination can replace the point-bivariate correlation coefficient, information gain, and univariate logistic regression coefficient; the confidence level can replace the Bayesian posterior update; and when the sample is sufficient, regularized multivariate fitting can be used, with three levels of protection as prior constraints.

[0025] Furthermore, in addition to manual operation, feedback tags can also incorporate online campaign metrics such as completion rate, like rate, and conversion rate, with thresholds set to distinguish between positive and negative samples.

[0026] Furthermore, version control can replace version vectors and configure hashes to achieve more granular cache invalidation.

[0027] An automatic video highlight segment extraction system based on multi-signal fusion and feedback self-calibration, used to implement the above method, is characterized by comprising: The 100 Media Parsing Module includes an audio track extraction unit and a shot segmentation unit; it is used to extract video audio tracks and output a shot boundary table. The 200-word text acquisition module includes an ASR recognition unit, an OCR subtitle recognition unit, and a dual-source alignment unit; it outputs a unified subtitle script that balances temporal accuracy and text accuracy. The 300 audio emotion module includes an intensity extraction unit, a robust normalization unit, and a rolling baseline bidirectional bias unit; it outputs the audio emotion intensity envelope. The 400 semantic candidate module includes a candidate generation unit and a cross-set memory injection unit; it is used to call the large model to output semantic highlight candidates and has a fallback function for model degradation. The 500 visual candidate module includes a native video input unit, a frame extraction and degradation unit, and an empty return fallback unit; it outputs visual pop candidate points, and multi-level degradation ensures that the main link is not interrupted. The 600 fusion module includes, in sequence, a threshold filtering unit, a duration constraint unit, a cut-point double adsorption unit, a deduplication and merging unit, a complementary addition unit, a structure prior unit, a weighted sorting unit, and a leaderboard output unit; it receives multiple candidates, performs deterministic fusion cut-point calculation, and outputs a highlight leaderboard. The 700-module output includes a vertical screen conversion unit, a copywriting generation unit, and a highlight compilation unit; the output can be directly used for short videos and highlight compilations. The 800 feedback acquisition module records manual film review operation events and outputs a feedback event stream; The 900 self-calibration module includes a discrimination calculation unit, a confidence mixing unit, and a floor protection unit; it receives feedback event streams, outputs updated weights, and feeds them back to the weighted sorting unit of the fusion module. Version 1000 and the caching module maintain the link version number, and version changes drive cache and historical ranking invalidation.

[0028] The text links (200+400), audio emotion module (300), and visual candidate module (500) are independent of each other, and all outputs are sent to the fusion module (600). If any single link fails, the fusion module can still output the highlight list by relying on the remaining links.

[0029] Compared with the prior art, the beneficial effects of the present invention are: 1. Improved recall coverage: Visual links fill in visual highlights without dialogue, and the audio bidirectional deviation mechanism captures silent highlights, solving the fundamental blind spots of pure semantic solutions; multiple signals are independent, and the failure of a single signal only reduces recall and will not cause overall collapse.

[0030] 2. Significantly improved cutting quality: Dual adsorption and rejection conditions ensure that dialogue is not cut off or the shot is not arbitrarily fragmented, solving the problem of sentence cutting and segment length distortion caused by unconditional alignment of shot boundaries.

[0031] 3. Intercept abnormal output from the model: use hard duration constraints to filter fragmented segments and excessively long segments; prioritize trimming excessively long segments to the hook point to adapt to the needs of short video traffic generation.

[0032] 4. Eliminate duplicate slices while retaining cross-modal confidence information: Perform deduplication after snapping to resolve collapsed duplication; when the same specular highlight is hit across modalities, add ranking points and do not add duplicate entries.

[0033] 5. Self-learning is safe and controllable, and small samples have a lower bound on the effect: three levels of protection including discrimination truncation, confidence mixing, and main signal floor; the system performance under small samples is no worse than manually set prior weights, self-calibration can be enabled immediately after going live, and the adoption rate continues to improve as samples accumulate, and the entire calculation process is interpretable and traceable.

[0034] 6. Results are reproducible and comparable: Link version number-driven caching and historical ranking invalidation ensure consistent ranking criteria and provide a clean dataset for weight self-calibration.

[0035] 7. Strong engineering robustness: Multi-stage and multi-level degradation fallback, fault tolerance for external dependencies of ASR, LLM, and multimodal models, suitable for industrial implementation. Attached Figure Description

[0036] Figure 1 This is a flowchart of the automatic video highlight segment extraction method in an embodiment of the present invention; In the diagram, S10 to S100 correspond one-to-one with the step numbers in the main text of the instruction manual; the top three columns are three parallel signal links that converge at S60; the dashed line on the left is the weight backfeed path (S90 → S60), and the dashed line on the right is the link version-driven failure path; Figure 2 This is a schematic diagram of the automatic video highlight segment extraction system in an embodiment of the present invention; The modules 100 to 1000 in the diagram correspond one-to-one with the module numbers in the main text of the instruction manual; the dashed lines represent the two control paths of recharge and failure, and the solid lines represent the data path. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example 1

[0038] An automatic method for extracting video highlight segments based on multi-signal fusion and feedback self-calibration is shown in Figure 1: Enter a short video file containing one episode of an animated series.

[0039] S10 Media Analysis: Extracts audio tracks, segments shots, and outputs the start and end times and shot boundary table for each shot.

[0040] S20 Text Acquisition Dual-Source Alignment: ASR recognizes the voiceover and outputs segmented subtitles with timestamps; OCR recognizes embedded subtitles within the screen. ASR provides a timestamp skeleton, and OCR corrects typos in the speech recognition, merging them to obtain a unified subtitle text.

[0041] S30 Audio Emotion Signal: Extracts RMS loudness and onset intensity, normalizes to the 95th percentile; calculates the rolling baseline in a 10-second window, and takes the absolute difference between the current audio value and the baseline as the main audio signal. High scores can be obtained for explosive fighting and suspenseful whispering segments, and the audio emotion envelope curve is output.

[0042] S40 Semantic Candidates: The unified subtitle script is input into the large language model, and the output highlights the candidate time period, type, semantic intensity score, and hook ending; the character and plot summary of this series is loaded as cross-episode memory; the backup model is automatically switched when the large model refuses to respond.

[0043] S50 Visual Candidates: The original video is transcoded into a low-resolution short video without audio, which is then fed into a multimodal model to output candidates for visual highlights such as fight scenes and transformations without dialogue, and outputs a visual impact score; if the inference times out, it is downgraded to extract keyframes; if the keyframe processing fails, it returns null without interrupting the main process.

[0044] S60 fusion cut point determination: The first step is to filter candidates with low semantic scores; excessively long fragments are cropped to hook points; double snapping is performed, iteratively aligning sentence boundaries, and then trying shot boundaries. Shot boundaries cannot cross sentences before replacement of cut points are allowed; excessively short fragments are discarded.

[0045] After adsorption is complete, perform IoU deduplication, repeat candidate grouping, and retain the candidate with the highest semantic value in each group.

[0046] If a group contains both semantic and visual source candidates, a visual enhancement is applied to the group representative.

[0047] Structural prior scoring is performed based on the position of the fragment in the whole set, with higher prior scores obtained in the 55-75% range.

[0048] The semantic score, audio score, and structural score are weighted and summed, and the highlight list is output in descending order.

[0049] The S70 generates a video, cuts segments, performs landscape to portrait mode conversion, and generates a title.

[0050] S80 operations staff manually review slices, marking them as accepted, rejected, or requiring minor adjustments, thus generating feedback events.

[0051] S90 Weight Self-calibration: Collect valid positive and negative samples, calculate the discrimination of each signal; combine sample size and manual prior to perform confidence mixing, and then perform semantic master signal floor protection; update the weight file, and automatically load the new weights for the next video processing.

[0052] S100 version control: Modify prompt words and fusion rules, increment version number by 1, invalidate old caches and old leaderboards, and recalculate all. Example 2

[0053] An automatic video highlight segment extraction system based on multi-signal fusion and feedback self-calibration is shown in Figure 2.

[0054] The system includes a media parsing module, a text acquisition module, an audio emotion module, a semantic candidate module, a visual candidate module, a fusion module, a final output module, a feedback acquisition module, a self-calibration module, and a version caching module.

[0055] The media parsing module extracts audio tracks and segments shots. The text acquisition module combines ASR and OCR to output high-quality subtitles with timestamps. The semantic module outputs semantic highlight candidates, the visual module outputs visual highlight candidates, and the audio module outputs audio scoring curves. All three independent candidates are fed into the fusion module, which performs threshold filtering, duration constraints, double segmentation, deduplication and merging, cross-modal enhancement, prior scoring, and weighted sorting to output a highlight list. The final video module outputs short video segments. The manual feedback collection module records operational feedback and sends it to the self-calibration module; the self-calibration module updates the weights after discrimination, confidence mixing, and floor protection, and then feeds it back into the fusion module. The version caching module maintains the link version number, and changes trigger cache invalidation.

[0056] This system supports the failure of any sub-module, and the remaining modules will still output highlight results.

[0057] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatic extraction of video highlight segments based on multi-signal fusion and feedback self-calibration, characterized in that, Includes the following steps: S10, Media Analysis: Extract the audio track from the input long video and perform shot segmentation to obtain the shot boundary table; S20. Text Acquisition and Dual-Source Alignment: Obtain timestamped text segments from the audio track through Automatic Speech Recognition (ASR), and recognize embedded subtitles in the video through OCR. Using ASR output as the timestamp skeleton and OCR output as the text truth value for error correction, the unified subtitle is obtained by aligning and merging them. S30. Audio Emotion Signal Extraction: Extract the root mean square energy and onset intensity curve of the audio track, and use the 95th quantile for robust normalization; calculate the absolute value of the deviation of the audio signal from the rolling baseline of this episode as the main audio signal, and output the audio emotion intensity envelope. S40, Semantic Candidate Generation: Input the unified subtitle script into the large language model and output the highlight candidate interval, semantic intensity score, and suggested time points for hook endings; set up a fallback for model downgrading; S50, Visual Candidate Generation: Transcode long videos into low-resolution audio-free videos, input the native multimodal model to obtain visual pop candidate; set up a three-level degradation of "native video input → extract keyframes → return empty candidates" so that the main process is not interrupted if the visual link fails; S60. Fusion and cut-off point determination: All steps employ deterministic rules; the model does not participate in cut-off point decisions. This includes the following sub-steps: S61. Filter candidates whose semantic strength is below the threshold; S62. Duration hard constraint: Overly long candidates are preferentially pruned to the suggested time point at the end of the hook; if there is no hook point, they are pruned to the maximum duration. S63, Dual Adsorption of Cutting Points: First, iteratively adsorb sentence boundaries; then try to adsorb shot boundaries, setting rejection conditions: shot boundaries must meet time direction requirements and shot boundaries must not cross any sentences before cutting point replacement is allowed; after adsorption is complete, discard fragment candidates with a duration lower than the lower limit. S64. After the tangent adsorption is completed, candidate groups are merged based on the intersection-union ratio (IoU) and each group retains the candidate with the highest semantic score as the group representative. S65. When the same candidate group contains both semantic source candidates and visual source candidates, the group representative candidate will receive a bonus that is positively correlated with the visual impact score, but no new slice entries will be added. S66. Perform structural position prior scoring based on the position of candidate segments in the entire film duration; S67. Sum the semantic score, audio score, and structural position score according to their weights, and output the highlight list in descending order of the final score; S70, Final Processing: Based on the highlight list, clips are cropped, horizontal to vertical screen conversion is performed, and short videos and highlights are generated; S80. Manual review feedback collection: Record manual operation events and distinguish between positive examples, negative examples, and weak positive examples; rewriting copy is not included in the weight calculation. S90, Self-calibration of fusion weights: S91, Filter valid tag events; S92. Calculate the discrimination of each signal. Set the discrimination to zero when it is negative and normalize it to obtain the learning weight. S93. Calculate the confidence level based on the effective sample size, and normalize the learning weights and artificial prior weights after mixing the confidence levels; skip calibration if either positive or negative sample is missing. S94. Set a lower limit for the weight of the main signal. If it is lower than the lower limit, raise it to the lower limit. The remaining weight is redistributed among the other signals. Write the updated weight to disk and back to step S67. S100, Link Version and Cache Failure Control: Maintain the link version number. When changes are made to the configuration of the highlight output results, the version number increments, triggering the automatic invalidation and rerun of intermediate cache and historical disk ranking lists.

2. The method according to claim 1, characterized in that, The prior scoring of the structural position, with the threshold for the time proportion of high partitions set to 0.65, is adapted to the distribution of short drama highlights concentrated in the 55%-75% time range of the entire film.

3. The method according to claim 1, characterized in that, Step S63 involves iteratively absorbing sentence boundaries: the starting point is the minimum start time of all sentences that cross the cutting point, and the ending point is the maximum end time of all sentences that cross the cutting point. This process is repeated until no sentences cross the cutting point.

4. The method according to claim 1, characterized in that, Step S93 uses the formula α=n / (n+K) to calculate the confidence level, where n is the number of valid samples and K is a smoothing constant. When the number of samples is small, the weights are biased towards the human prior. The sample size is sufficient, and the weights are biased towards the data learning results.

5. The method according to claim 1, characterized in that, Step S30 uses the rolling mean of a 10-second window for the rolling baseline; high-decibel bursts of audio and sudden silences can both yield higher audio emotion scores.

6. The method according to claim 1, characterized in that, It also includes cross-episode memory steps: maintaining the series character list and main plot scrolling summary, inputting a large model to identify cross-episode foreshadowing, and updating cross-episode memory after processing a single episode.

7. The method according to claim 1, characterized in that, It can increase the density of bullet comments, facial expressions, and motion vector signal channels, or remove audio links when there are no audio tracks; sentence boundaries are replaced with speech endpoints to detect silence intervals, and camera boundaries are replaced with music beat points, while the rejection condition logic remains unchanged.

8. The method according to claim 1, characterized in that, Feedback tags can incorporate online campaign metrics such as completion rate, like rate, and traffic conversion data. Thresholds can be set to distinguish between positive and negative samples, while the rest of the process remains unchanged.

9. An automatic video highlight segment extraction system based on multi-signal fusion and feedback self-calibration, used to implement the method described in any one of claims 1-8, characterized in that, include: The media parsing module is used to extract audio tracks, perform shot segmentation, and output a shot boundary table. The text acquisition module includes an ASR recognition unit, an OCR recognition unit, and a dual-source alignment unit, which outputs a unified subtitle. The audio emotion module is used to extract audio features, perform robust normalization, calculate the bidirectional bias relative to the rolling baseline, and output the audio emotion envelope. The semantic candidate module is used to call the large model to output semantic highlight candidates and has a fallback function for model degradation. The visual candidate module performs multimodal inference on native video input, and has three levels of degradation: native video, frame extraction, and empty return, and outputs visual highlight candidates. The fusion module includes a threshold filtering unit, a duration constraint unit, a cut-point double adsorption unit, a deduplication and merging unit, a complementary addition unit, a structure prior unit, a weighted sorting unit, and a leaderboard output unit. It receives multiple candidate signals, performs deterministic fusion, calculates cut-points, and outputs a highlight leaderboard. The final video module is used for video cropping, landscape to portrait conversion, text generation, and highlight splicing. The feedback collection module records the feedback event stream output by the manual film review process; The self-calibration module includes a discrimination calculation unit, a confidence mixing unit, and a floor protection unit; it receives feedback event streams and outputs updated weights to feed back into the fusion module. The version and caching module maintains the version number of the link, and version updates drive caching and invalidation of historical rankings; Among them, the text link, audio emotion module, and visual candidate module independently output candidates and send them to the fusion module; if any link fails, the fusion module still outputs the highlight list by relying on the remaining links.

10. The system according to claim 9, characterized in that, The system can be applied to scenarios involving converting long videos into short videos and highlighting them in animation short dramas, film and television dramas, variety shows, sports events, live broadcast replays, teaching videos, and meeting recordings.

Images