A video acceleration processing method

Abstract

This invention discloses a video acceleration processing method, relating to the field of video processing technology. It involves acquiring image data, audio data, and speech text, extracting corresponding semantic factors, constructing a joint semantic representation vector, calculating burst intensity and generating a semantic burst curve, obtaining user preferences and generating a rhythm control strategy, executing accelerated playback and retaining key content according to the strategy, generating skip transition frames to enhance video continuity, encapsulating the processed video, and generating auxiliary index data. This invention performs video acceleration processing based on multimodal semantic burst analysis, accurately identifying key content and generating a rhythm control strategy based on user preferences. Simultaneously, the generation of transition frames enables natural transitions between skip segments, improving the efficiency and continuity of video playback.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a video acceleration processing method. Background Technology

[0002] With the increasing prevalence of long-form video data in scenarios such as remote education, smart conferencing, security monitoring, and content browsing, users' demand for "efficiently acquiring key information" is constantly increasing. Existing video acceleration technologies generally adopt fixed-speed playback or skipping methods based on camera cuts, lacking an understanding of the semantic content of the video.

[0003] Currently, Chinese patent application number CN202111528704.2 discloses an accelerated processing method for a video compression card based on a RISC-V processor, belonging to the technical field of video compression card software compression research. The method executes the following steps: 1) acquiring image information; 2) performing AVS format encoding on the image information; 3) using the interaction between the RISC-V processor and the coprocessor to complete zig-zag scanning and run-length encoding of the image information in step 2); 4) performing AVS format entropy encoding on the image information that has undergone zig-zag scanning and run-length encoding in step 3); 5) completing AVS video compression; the coprocessor participates in the execution, memory access, and write-back functions of the RISC-V processor through coprocessor instructions.

[0004] The relevant technologies struggle to dynamically identify key segments based on video semantic content and achieve personalized rhythm control and natural jump transitions. Summary of the Invention

[0005] The technical problem solved by this invention is that existing technologies are unable to dynamically identify key segments based on video semantic content and achieve personalized rhythm control and natural skipping transitions.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A video acceleration processing method includes the following steps:

[0008] Step S1: Collect image data, audio data, and speech text, and extract the corresponding semantic factors;

[0009] Step S2: Construct a joint semantic representation vector, calculate the burst intensity, and generate a semantic burst curve;

[0010] Step S3: Obtain user preferences and generate rhythm control strategy;

[0011] Step S4: Implement accelerated playback and retain key content according to the strategy;

[0012] Step S5: Generate skip transition frames to enhance video continuity;

[0013] Step S6: Encapsulate the processed video and generate auxiliary index data;

[0014] Step S1 includes the following sub-steps:

[0015] Step S101: Extract continuous frame image data from the original video stream and analyze the dynamic features of the screen based on the inter-frame changes. The analysis of the dynamic features of the screen includes calculating the pixel change ratio of adjacent frames to determine the shot switching rate, tracking the motion path of the center point of the screen and the main object to determine the screen movement amplitude, applying a target detection model to identify the main content of the video and record its frequency and duration of occurrence, and constructing an image change sequence.

[0016] Step S102: Extract speech signals from audio data, analyze speech rate, intonation changes and stressed word features, and transcribe the audio data into text to generate a speech-text sequence with timestamps;

[0017] Step S103: Process the speech-text sequence, perform text normalization, sentence segmentation, semantic word segmentation and keyword extraction, and use syntactic analysis tools to identify subject-predicate structure and modification relationship, and output semantic structure labels and keyword sequence;

[0018] Step S2 includes the following sub-steps:

[0019] Step S201: Align the image change sequence, speech text sequence, semantic structure label and keyword sequence in the time dimension, synchronize the continuous frame image data, audio data and speech text sequence with a unified time axis as the reference, and construct a joint semantic representation vector in each time window. The joint semantic representation vector includes the degree of visual subject change, speech rate and emotional fluctuation intensity, keyword density and semantic mutation degree.

[0020] Step S202: Perform sliding time window analysis on the joint semantic representation vector, calculate the semantic burst intensity score, identify burst or turning semantic segments by detecting peaks and trend changes, and classify them into structural nodes and core nodes. Output the semantic change trajectory obtained by trend modeling as a semantic burst curve.

[0021] Step S3 includes the following sub-steps:

[0022] Step S301: Obtain the user's preference settings for playback rhythm, including the expected total duration, the type of content to be followed, and the minimum retention ratio, and convert them into structured parameters;

[0023] Step S302: Combining the semantic burst curve and structured parameters, a rhythm control strategy is generated. By analyzing the burst intensity and user attention type matching in each time period within the semantic burst curve, playback rate and skipping markers are allocated, and a rhythm strategy file is output.

[0024] Step S4 includes the following sub-steps:

[0025] Step S401: Accelerate the playback of video segments of unstructured nodes and core nodes, compress the time length through image resampling, retain key images, perform temporal reconstruction, and align speech rate changes with text.

[0026] Step S402: Retain the video segments of structural nodes and core nodes, and perform slow-down processing according to user preferences;

[0027] Step S5 includes the following sub-steps:

[0028] Step S501: Extract keyframe images with content continuity features from the video segments marked by skipping, perform time alignment, calibrate hue, brightness and motion direction, and establish a screen transition interface.

[0029] Step S502: Based on the video transition interface, extract the image features and corresponding semantic information of the video segments before and after the skip mark, and generate a transition video frame sequence;

[0030] The construction of the transition video frame sequence combines the image color distribution, subject edge contour, background texture changes and semantic tag continuity of the video segments before and after the jump mark. The frame interpolation method is used to generate inter-frame transition images, and the audio segments are processed for duration padding and intonation smoothing.

[0031] Step S6 includes the following sub-steps:

[0032] Step S601: Encapsulate the processed video content and output a complete video file, wherein the complete video file includes an image stream, a text track, and an audio track;

[0033] Step S602: Generate auxiliary index data, which includes structural nodes, core node timestamps, start and end times of reserved segments, playback rate, skipping range, chapter numbers, semantic tags, and keyword summaries.

[0034] Preferably, step S202 identifies the starting point of a new topic and the climax of an event based on the sudden intensity change trend, and accordingly increases the corresponding rhythm weight, prioritizing or slowing down the playback in rhythm control.

[0035] Preferably, in step S302, during the generation of the rhythm control strategy, the playback rate and skipping range of low-priority segments are adjusted according to the user's time budget while retaining key semantic content.

[0036] Preferably, in step S502, when generating the virtual transition screen, the image change sequence, background motion and subject outline are integrated, image interpolation and deformation control methods are applied, and edge alignment and inter-frame smoothing are performed.

[0037] The beneficial effects of this invention are as follows: This invention uses multimodal semantic burst analysis for video acceleration processing, which can accurately identify key content and generate rhythm control strategies in combination with user preferences. At the same time, it achieves natural connection of skip segments through transition frame generation, thereby improving the efficiency and continuity of video playback. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the steps of a video acceleration processing method according to an embodiment of the present invention. Detailed Implementation

[0039] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0040] Example, refer to Figure 1 A video acceleration processing method is provided, including the following steps:

[0041] Step S1: Collect image data, audio data, and voice text, and extract the corresponding semantic factors.

[0042] Step S2: Construct a joint semantic representation vector, calculate the burst intensity, and generate a semantic burst curve.

[0043] Step S3: Obtain user preferences and generate rhythm control strategies.

[0044] Step S4: Implement accelerated playback and retain key content according to the strategy.

[0045] Step S5: Generate skip transition frames to enhance video continuity.

[0046] Step S6: Encapsulate the processed video and generate auxiliary index data.

[0047] Step S1 includes the following sub-steps:

[0048] Step S101: Extract continuous frame image data from the original video stream, and analyze the dynamic features of the screen based on the inter-frame changes. The analysis of the dynamic features of the screen includes calculating the pixel change ratio of adjacent frames to determine the shot switching rate, tracking the motion path of the center point of the screen and the main object to determine the screen movement amplitude, applying the object detection model to identify the main content of the video and recording its frequency and duration, and constructing an image change sequence.

[0049] Step S101 is used to extract and quantify the features of image changes between video frames. By constructing image change sequences through elements such as shot switching rate, subject movement path and object appearance frequency, a dynamic basis is provided for visual perception in semantic bursts.

[0050] Step S102: Extract speech signals from audio data, analyze speech rate, intonation changes and stressed word features, and transcribe the audio data into text to generate a speech-text sequence with timestamps.

[0051] Step S102 is used to extract speech variation features such as speech rate and intonation from the audio data and transcribe them into a time-stamped speech text sequence to provide acoustic information support for subsequent semantic analysis and multimodal alignment.

[0052] Step S103: Process the speech-text sequence, perform text normalization, sentence segmentation, semantic word segmentation and keyword extraction operations, and use syntactic analysis tools to identify subject-predicate structure and modification relationship, and output semantic structure labels and keyword sequence.

[0053] Step S103 is used to perform linguistic processing on the speech text sequence, extract keywords and grammatical structures, and output a structured set of semantic tags as important semantic factors for constructing a joint semantic representation vector.

[0054] Step S1 is used to extract three basic modal data—image, audio, and text—from the video and construct a multimodal semantic factor set to provide complete, time-aligned semantic basic data for subsequent semantic burst analysis and rhythm control strategy generation.

[0055] Step S2 includes the following sub-steps:

[0056] Step S201: Align the image change sequence, speech text sequence, semantic structure label and keyword sequence in the time dimension. Synchronize the continuous frame image data, audio data and speech text sequence with a unified time axis as the reference. Construct a joint semantic representation vector within each time window. The joint semantic representation vector includes the degree of visual subject change, speech rate and emotional fluctuation intensity, keyword density and semantic mutation degree.

[0057] Step S201 is used to align the image change sequence, speech text sequence and semantic label information along a unified time axis, synchronize image, audio and text features, and construct a joint semantic representation vector containing information such as visual changes, speech rate and emotion changes and text keyword mutations, so as to provide a unified input data format for subsequent burst detection.

[0058] Step S202: Perform sliding time window analysis on the joint semantic representation vector, calculate the semantic burst intensity score, identify burst or turning semantic segments by detecting peaks and trend changes, and classify them into structural nodes and core nodes. The semantic change trajectory obtained by trend modeling is output as a semantic burst curve.

[0059] Step S202 identifies the starting point of new topics and the climax of events based on the sudden intensity change trend, and increases the corresponding rhythm weight accordingly, prioritizing or slowing down the playback in rhythm control.

[0060] Step S202 is used to perform sliding window analysis on the joint semantic representation vector, calculate the semantic burst intensity score for each time period, identify time segments with significant semantic changes, and divide them into structural nodes and core nodes. At the same time, the semantic burst curve is output through trend modeling, which serves as the core basis for content layering and rhythm labeling. When a trend of significant increase in burst intensity is detected, the starting point of a new topic or the climax of an event is identified, and the rhythm weight of that time period in the rhythm strategy is increased accordingly to ensure that such segments are given priority to be retained or played at a slower pace, thereby ensuring the comprehensibility of key content and the continuity of rhythm.

[0061] Step S2 is used to fuse image, speech and text features to construct a joint semantic representation vector that reflects the trend of multimodal change, and to identify semantic burst segments based on its time series changes, generating semantic burst curves to provide a structured semantic basis for rhythm regulation strategies.

[0062] Step S3 includes the following sub-steps:

[0063] Step S301: Obtain the user's preference settings for playback rhythm. The preference settings include the expected total duration, the type of content to be focused on, and the minimum retention ratio, and convert them into structured parameters.

[0064] Step S301 is used to collect users' personalized needs regarding playback rhythm, including the expected total video duration, the types of content they are interested in, and the minimum acceptable proportion of content to be retained. This information is then converted into structured parameters to provide constraints and priorities for strategy generation.

[0065] Step S302: Combining the semantic burst curve and structured parameters, a rhythm control strategy is generated. By analyzing the matching between the burst intensity and user attention type in each time period within the semantic burst curve, playback rate and skipping markers are allocated, and a rhythm strategy file is output.

[0066] In step S302, during the generation of the rhythm control strategy, the playback rate and skipping range of low-priority segments are adjusted according to the user's time budget while retaining key semantic content.

[0067] Step S302 is used to integrate semantic burst curves and structured user preference parameters to identify time periods with high semantic intensity and that match the user's attention type, generate a rhythm control strategy, assign corresponding playback rate and skip mark to each segment, and form a rhythm strategy file for subsequent modules to call. In the process of generating the rhythm control strategy, the playback rate and skip interval of low priority segments are dynamically adjusted to ensure that the final playback duration meets the user's budget, and key semantic content is retained first during compression to ensure that the core information is not omitted.

[0068] Step S3 is used to generate a rhythm control strategy based on the playback preference parameters set by the user and combined with the semantic burst curve, so as to complete the intelligent allocation of playback rate and skipping range for each time period and realize personalized playback rhythm control.

[0069] Step S4 includes the following sub-steps:

[0070] Step S401: Accelerate the playback of video segments of unstructured nodes and core nodes, compress the time length through image resampling, retain key images, perform temporal reconstruction, and align speech rate changes with text.

[0071] Step S401 is used to perform acceleration processing on ordinary segments that are not marked as structural nodes or core nodes. Specifically, it compresses the time length by resampling the image, while retaining key frame images and reconstructing the image order. It also aligns speech rate changes with the text to avoid gaps in content comprehension.

[0072] Step S402: Retain the video segments of structural nodes and core nodes, and perform slow-down processing according to user preferences.

[0073] Step S402 is used to keep the important semantic segments corresponding to the structural nodes and core nodes playing in their entirety, and to perform an appropriate slowdown operation according to user preferences, so as to make the key content more fully displayed and enhance the viewer's ability to absorb key information.

[0074] Step S4 is used to perform video acceleration and key segment retention operations according to the rhythm control strategy, compress the overall playback time through differentiated playback strategy, and at the same time ensure that key content is clearly presented and semantic logic is not destroyed.

[0075] Step S5 includes the following sub-steps:

[0076] Step S501: Extract keyframe images with content continuity features from the video segments marked by skipping, perform time alignment, calibrate hue, brightness and motion direction, and establish a screen transition interface.

[0077] Step S501 is used to extract keyframe images of two retained segments before and after the skipped video segment. These keyframes have content continuity characteristics. By aligning the keyframes in time and calibrating their color, brightness, and motion direction, a stable transition interface is established, providing image anchors for the insertion of subsequent transition frames.

[0078] Step S502: Based on the video transition interface, extract the image features and corresponding semantic information of the video segments before and after the skip mark, and generate a transition video frame sequence.

[0079] The construction of the transition video frame sequence combines the image color distribution, subject edge contour, background texture changes and semantic tag continuity of the video segments before and after the jump mark. The frame interpolation method is used to generate inter-frame transition images, and the audio segments are processed for duration padding and intonation smoothing.

[0080] In step S502, when generating the virtual transition screen, the image change sequence, background motion and subject outline are integrated, image interpolation and deformation control methods are applied, and edge alignment and inter-frame smoothing are performed.

[0081] Step S502 is used to extract video image features and their corresponding semantic information before and after the skipped segment based on the screen transition interface, and generate a sequence of transitional video frames. This sequence comprehensively considers the color distribution, subject edge contour, background texture changes and semantic tag continuity of the frames before and after the skip, and uses a frame interpolation algorithm to generate visual connection frames. At the same time, it fills in the audio gaps caused by the skip and smooths the intonation to improve auditory consistency. In the process of generating virtual transition frames, it further integrates image change sequences, background dynamic features and subject shape contours, and applies image interpolation and deformation control methods to make the inserted frames consistent with the original segments in style, structure and dynamics. And through edge alignment and inter-frame smoothing operations, it achieves a natural visual transition of the video.

[0082] Step S5 is used to construct image frames and audio streams with natural transitions before and after the skipped segments, solving the visual and auditory abruptness caused by skipping and achieving a smooth playback experience for accelerated videos.

[0083] Step S6 includes the following sub-steps:

[0084] Step S601: Encapsulate the processed video content and output a complete video file, which includes an image stream, a text track, and an audio track.

[0085] Step S601 is used to encapsulate and output the video content after acceleration processing, skipping transition compensation and rhythm strategy execution, to generate a complete video file containing image stream, synchronized text track and audio track, to ensure that multimodal data is synchronized and consistent, and to adapt to the playback requirements of multiple terminals.

[0086] Step S602: Generate auxiliary index data, which includes structural nodes, core node timestamps, start and end times of reserved segments, playback rate, skipping range, chapter numbers, semantic tags, and keyword summaries.

[0087] Step S602 is used to generate auxiliary index data corresponding to the processed video content. The index information includes timestamps of structural nodes and core nodes, start and end times of retained segments, playback rates of each segment, range of skipped segments, chapter numbers, semantic tags and keyword summaries, which can be used for users to quickly navigate, for the system to perform secondary analysis or for interface linkage calls.

[0088] Step S6 is used to output the accelerated video file and its structured auxiliary index data, so as to realize the efficient delivery and navigable management of video content, and improve the user playback experience and system integration capabilities.

[0089] This invention constructs semantic burst curves through multimodal fusion of images, audio, and text, effectively identifying key segments such as new topics and climaxes. It automatically generates rhythm control strategies based on the user's set time budget and focus type, achieving differentiated playback speed and skip marker allocation. By extracting key frames before and after skips and generating transition frame sequences, it achieves natural visual, auditory, and subtitle connections between skip segments, generating auxiliary index data, including structural nodes and semantic tags, to facilitate user navigation, skipping, and review. This method is not only suitable for educational review and meeting minutes, but can also be used in scenarios such as content summary generation and mobile video browsing.

[0090] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for video acceleration processing, the method comprising: The method comprises the following steps: Step S1, collecting image data, audio data and speech text, and extracting corresponding semantic factors; Step S2, constructing a joint semantic representation vector, calculating burst intensity and generating a semantic burst curve; Step S3, obtaining user preferences and generating a rhythm control strategy; Step S4, performing accelerated playback and key content preservation according to the strategy; Step S5, generating a skip transition frame to enhance video continuity; Step S6, packaging the processed video and generating auxiliary index data; The step S1 comprises the following sub-steps: Step S101, extracting continuous frame image data from the original video stream, and analyzing the picture dynamic characteristics based on the inter-frame change, which includes calculating the pixel change ratio of adjacent frames to determine the lens switching rate, tracking the motion path of the picture center point and the main object to determine the picture movement amplitude, applying a target detection model to identify the main content of the video and record its appearance frequency and duration, and constructing an image change sequence; Step S102, extracting speech signals from the audio data, analyzing speech speed, tone change and repeated word features, and transcribing the audio data into text to generate a speech text sequence with timestamps; Step S103, processing the speech text sequence, performing text normalization, sentence segmentation, semantic word segmentation and keyword extraction operations, and using a syntax analysis tool to identify subject-predicate structures and modification relationships, and outputting semantic structure labels and keyword sequences; The step S2 comprises the following sub-steps: Step S201, aligning the image change sequence, speech text sequence, semantic structure labels and keyword sequence in the time dimension, synchronizing the continuous frame image data, audio data and speech text sequence based on the unified time axis, constructing a joint semantic representation vector in each time window, and the joint semantic representation vector includes visual subject change degree, speech speed and emotional fluctuation intensity, keyword density and semantic mutation degree; Step S202, performing sliding time window analysis on the joint semantic representation vector, calculating the semantic burst intensity score, identifying the burst or turning point semantic section by detecting the peak value and trend mutation, and classifying it into structure nodes and core nodes, and outputting the semantic burst curve through the semantic change trajectory obtained by trend modeling; The step S3 comprises the following sub-steps: Step S301, obtaining user preference settings for playback rhythm, including expected total length, content type of interest and minimum retention ratio, and converting them into structured parameters; Step S302, generating a rhythm control strategy based on the semantic burst curve and structured parameters, assigning playback speed and skip markers by analyzing the burst intensity and user interest type matching in each time period of the semantic burst curve, and outputting a rhythm strategy file; The step S4 comprises the following sub-steps: Step S401, accelerating the playback of video segments of non-structure nodes and core nodes by image resampling to compress the time length, retaining key pictures and performing time sequence reconstruction and aligning the speech speed change and text; Step S402, retaining the video segments of structure nodes and core nodes, and performing deceleration processing according to user preferences; The step S5 comprises the following sub-steps: Step S501, extracting key frame images with content continuity features from the skip-marked video segments, and performing timing alignment, calibrating color, brightness and motion direction, and establishing picture transition interface; Step S502, based on the picture transition interface, extracting image features and corresponding semantic information of the video segments before and after the skip mark, and generating transition video frame sequence; The construction of the transition video frame sequence combines the image color distribution, main body edge contour, background texture change and semantic label continuity of the video segments before and after the skip mark, generates inter-frame transition images in the interpolation mode, and performs time length complementation and tone smoothing processing on the audio segments; The step S6 includes the following sub-steps: Step S601, packaging the processed video content, and outputting a complete video file, the complete video file including image stream, text track and audio track; Step S602, generating auxiliary index data, the auxiliary index data including structure node, core node timestamp, reserved segment start and end time, play rate, skip range, chapter number, semantic label and keyword abstract.

2. The method of claim 1, wherein, The step S202 identifies the new topic starting point and event climax segment based on the burst intensity change trend, and correspondingly increases the rhythm weight, and preferentially retains or slows down the play in rhythm control.

3. The method of claim 2, wherein the video acceleration processing method is characterized by, The step S302 adjusts the play rate and skip range of the low-priority segment and retains the key semantic content according to the user time budget in the process of generating the rhythm control strategy.

4. The method of claim 3, wherein the video acceleration processing method is characterized by, The step S502 fuses image change sequence, background dynamic and main body contour when generating virtual transition picture, applies image interpolation and deformation control method, and performs edge alignment and inter-frame smoothing processing.

Images