Video progress bar interaction method and device, electronic equipment and storage medium

Abstract

The invention provides a video progress bar interaction method and device, electronic equipment and a storage medium, relates to the technical field of computers, in particular to the fields of multimedia content analysis, media navigation control interaction and the like, and can be used for application scenes such as video progress bar display and the like. The specific scheme is as follows: in response to a selection instruction for a target video clip, dividing the target video clip into a plurality of time intervals, and determining a content importance score of each time interval; performing non-uniform time scaling processing on the target video clip based on the content importance score; remapping and displaying a section, corresponding to the target video clip, in the progress bar according to the time proportion after the scaling processing; based on the progress bar, providing a frame-level positioning operation; and in response to a marking instruction triggered when the positioning of the target frame is completed through the frame-level positioning operation, generating marking information. According to the scheme, the time axis precision can be adjusted according to the content importance, and the operation step of locking the target frame is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of multimedia content analysis and media navigation control interaction, which can be used in application scenarios such as video progress bar display. Specifically, it relates to interactive methods, devices, electronic devices and storage media for video progress bars. Background Technology

[0002] In video playback and interaction, progress bars typically use a linear mapping, where a unit distance corresponds to a fixed duration. This approach is suitable for short videos, but in long videos, due to limited spatial resolution, users often need to repeatedly and precisely drag and pause to locate a specific keyframe, resulting in insufficient positioning accuracy, low operational efficiency, and a cumbersome process. Summary of the Invention

[0003] This disclosure provides an interactive method, apparatus, electronic device, and storage medium for a video progress bar.

[0004] According to a first aspect of this disclosure, an interactive method for a video progress bar is provided, comprising: in response to a selection instruction for a target video segment, dividing the target video segment into multiple time intervals and determining a content importance score for each time interval; performing non-uniform time scaling on the target video segment based on the content importance scores of each time interval, wherein the time interval with a higher content importance score is allocated a longer playback duration; remapping and displaying the segments in the progress bar corresponding to the target video segment according to the scaled time ratio; providing a frame-level positioning operation based on the progress bar; and generating marking information indicating the position of the target frame in response to a marking instruction triggered when the positioning of a target frame is completed through the frame-level positioning operation.

[0005] According to a second aspect of this disclosure, an interactive device for a video progress bar is provided, comprising: a scoring determination module, configured to, in response to a selection instruction for a target video segment, divide the target video segment into multiple time intervals and determine a content importance score for each time interval; a video scaling module, configured to perform non-uniform time scaling processing on the target video segment based on the content importance scores of each time interval, wherein time intervals with higher content importance scores are allocated a longer playback duration; a scaling adjustment module, configured to remap and display segments in the progress bar corresponding to the target video segment according to the scaled time ratio; a positioning provision module, configured to provide frame-level positioning operations based on the progress bar; and a marker generation module, configured to, in response to a marker instruction triggered when positioning a target frame is completed through frame-level positioning operations, generate marker information indicating the position of the target frame.

[0006] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the methods described in the embodiments of this disclosure.

[0007] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0008] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0009] By adopting the solution disclosed herein, the timeline accuracy can be dynamically adjusted according to the importance of the video content, and this adjustment can be combined with the progress bar display and frame-level positioning operation. This fundamentally changes the traditional linear and rigid interaction mode of the progress bar, achieves an order-of-magnitude improvement in the positioning accuracy of key content areas of the video, and significantly reduces the operation steps required to accurately lock the target frame.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is a flowchart illustrating the interaction method of a video progress bar according to an embodiment of the present disclosure; Figure 2 This is a schematic diagram of the structure of an interactive device for a video progress bar according to an embodiment of the present disclosure; Figure 3 This is a schematic diagram of a scenario for an interactive method of a video progress bar according to an embodiment of the present disclosure; Figure 4 This is a structural diagram of an electronic device used to implement the interactive method of the video progress bar in the embodiments of this disclosure. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] Before introducing the technical solutions of the embodiments of this disclosure, let's further explain the technical terms that may be used in this disclosure: A video progress bar refers to a timeline control in a video player interface, used to display the current playback progress and total duration, and allowing users to jump to different points on the timeline by clicking or dragging. The video progress bar establishes a linear mapping between spatial positions on the screen and time points in the video, meaning that the same unit distance corresponds to a fixed duration. Specifically, the video progress bar can also display auxiliary information such as buffering progress, current progress, and thumbnail previews.

[0014] In related technologies, video progress bars serve as a core control for locating and browsing video content. They compress the entire timeline to a limited screen length and map spatial locations to specific moments in a linear fashion. Under this linear model, short videos, due to their limited total duration, have a smaller time frame per pixel, resulting in acceptable addressing accuracy. However, for long videos, the same pixel distance covers a longer time interval, significantly reducing control resolution and making it difficult for users to precisely stop at the target moment or keyframe. Lacking semantic tagging and multi-scale navigation capabilities, users can only gradually approach their target through small drags, repeated attempts, and frequent pauses, leading to large addressing errors, high cognitive and operational burdens, low efficiency, and a limited overall interactive experience.

[0015] To at least partially address one or more of the aforementioned problems and other potential issues, this disclosure proposes an interactive method for video progress bars. By dynamically adjusting the timeline precision based on the importance score of each video frame, and combining this adjustment with progress bar display and frame-level positioning operations, this fundamentally changes the traditional linear and rigid interactive mode of progress bars, achieves precise positioning of key content areas in the video, and significantly reduces the number of operation steps required to accurately lock onto the target frame.

[0016] This disclosure provides an interactive method for a video progress bar. Figure 1This is a flowchart illustrating a video progress bar interaction method according to an embodiment of the present disclosure. This video progress bar interaction method can be applied to an interaction device for the video progress bar. The interaction device for the video progress bar is located in an electronic device. The electronic device includes, but is not limited to, fixed devices and / or mobile devices. For example, fixed devices include, but are not limited to, servers, which can be cloud servers or ordinary servers. For example, mobile devices include, but are not limited to, video progress bar display devices, which can be mobile phones, tablets, in-vehicle terminals, etc. In some possible implementations, the video progress bar interaction method can also be implemented by a processor calling computer-readable instructions stored in memory. Figure 1 As shown, the interaction methods for this video progress bar include: S101. In response to the selection instruction for the target video segment, the target video segment is divided into multiple time intervals, and the content importance score of each time interval is determined.

[0017] S102. Based on the content importance score of each time interval, perform non-uniform time scaling on the target video segment, where the time interval with the higher content importance score is assigned a longer playback duration.

[0018] S103. Remap and display the segment in the progress bar that corresponds to the target video clip according to the time ratio after scaling.

[0019] S104. Provides frame-level positioning operations based on the progress bar.

[0020] S105. In response to a marking instruction triggered when a target frame is located through a frame-level positioning operation, mark information is generated to indicate the position of the target frame.

[0021] Here, a selection command refers to an operation command used to trigger processing of a target video segment. It can be generated by user interaction within the interface. Upon receiving this command, the time range of a target video segment can be uniquely determined. A time interval refers to several consecutive segments on the timeline, divided sequentially from the determined target video segment. Each time interval has a clear start and end time, and the time intervals do not overlap in time, and when strung together, they exactly cover the entire target video segment. For example, the division can use a fixed duration or adaptive division based on scene transitions, shot boundaries, or semantic segments. Content importance score is a numerical representation of the content-level importance of each time interval. It can be calculated based on a combination of features, such as image motion intensity, scene change frequency, face or key object detection results, keyword density in audio / subtitles, and historical user dwell or rewatch behavior within that interval. Specifically, the score value can typically be set as a weight within a certain numerical range; a larger value indicates that the time interval is considered more important to the user's viewing experience or business objectives.

[0022] In this embodiment, the system first responds to a user's selection command on the playback interface, determining the start and end times of the target video segment corresponding to the selection command, thereby obtaining the time range of the target video segment. Subsequently, the time range can be divided according to a preset segmentation strategy to obtain multiple time intervals in chronological order. Then, after obtaining each time interval, the audio and video content of each time interval can be further analyzed, extracting various features including the degree of image change, speech semantic information, detected key events, and historical user interaction data. These features are then input into a preset model for calculation, thereby obtaining a content importance score corresponding to each time interval, and establishing a correlation between this score and the corresponding time interval.

[0023] Here, time scaling refers to the process of applying different time scaling ratios to different time intervals based on their content importance scores, while maintaining the internal time order of the target video segment. This results in a non-uniform adjustment of the actual playback rhythm, allowing for extended playback of high-importance intervals and compressed playback of low-importance intervals, thus achieving non-linear deformation of the timeline within the overall length constraint. Playback duration refers to the actual playback time allocated to a specific time interval after time scaling. In this embodiment, the time interval occupies time from the start to the end of playback. The playback duration is determined by the original duration of the interval and its content importance score; a higher importance score results in a longer allocated playback duration, and vice versa.

[0024] In this embodiment, the scores for all time intervals are first normalized, converting them into a set of weighted values ​​that can be used to allocate playback duration. Then, considering the target total duration of the target video segment within the overall playback process, the playback duration that each time interval should receive is calculated according to the weight ratio of each time interval, ensuring that time intervals with higher content importance scores are allocated longer playback durations and time intervals with lower scores are allocated shorter playback durations. Next, based on the ratio of the original duration to the playback duration of the interval, the playback speed or time scaling factor for that interval can be derived, forming a mapping relationship between timelines.

[0025] In this embodiment, the proportion of display length that each time interval should occupy on the progress bar can be calculated first, based on the proportion of the playback duration of each time interval within the target video segment to the total playback duration of the entire target video segment. Then, the continuous segment corresponding to the target video segment on the progress bar can be further subdivided into multiple sub-segments according to the aforementioned proportion, and the start and end positions of each sub-segment can be bound to the original time range of its corresponding time interval. Finally, the display of the progress bar segment can be redrawn or updated, so that time intervals with higher content importance scores occupy a more prominent or wider display area on the progress bar, while intervals with lower content importance scores occupy a narrower area. This creates a non-linear mapping on the progress bar consistent with the expanded and resized time proportions, allowing users to more intuitively perceive and access important content through progress bar interaction.

[0026] Here, frame-level positioning refers to the positioning operation where the user, based on the progress bar interface that has already undergone non-uniform time mapping, precisely resolves the user's operation to a specific video frame by dragging the progress bar slider, clicking a progress position, using the precise control buttons to move forward / backward one frame, or by zooming in on the timeline for fine dragging.

[0027] In this embodiment, when a user drags, clicks, or fine-tunes the progress bar, the corresponding progress bar position is first captured. Based on the non-linear mapping between the progress bar position and the stretched timeline, this position is converted into a specific time point on the stretched timeline. Subsequently, the mapping between the stretched timeline and the original video timeline is used to calculate the corresponding original video timestamp. Based on this timestamp and the video's frame rate information, the closest target frame number is deduced. Finally, the video decoding and rendering module is driven to jump to and display this frame, while simultaneously updating the corresponding preview or static screenshot on the interface. This allows the user to precisely locate video content at a single-frame granularity through progress bar operations.

[0028] Here, the target frame refers to the specific video frame currently displayed and confirmed by the user after completing the aforementioned frame-level positioning operation. It corresponds to a unique timestamp and frame number on the video timeline and is the object frame that the user wishes to mark or perform subsequent operations on. The marking instruction refers to the operation command issued by the user after locating the target frame in order to record, annotate, or mark that frame. Examples include clicking the "Add Mark" button, long-pressing the current frame on the screen, or triggering the marking function via a shortcut key. The marking information refers to the data set used to indicate the position and related attributes of the target frame in the video. This may include the original timestamp or frame number corresponding to the target frame, as well as visual attributes displayed on the progress bar. It may also include the marking type, user-filled text descriptions, associated tags or categories, etc., for use and display during subsequent playback and editing.

[0029] In this embodiment, the original video timestamp and frame number corresponding to the current playback position can be read first, and a new tag data record can be generated. Specifically, during the generation of tag data, the user-inputted notes, selected tag categories, and preset display styles can be simultaneously collected. This information is then encapsulated into a complete tag information object, which is stored in a tag list associated with the video or in a background database. Subsequently, based on the non-uniformly mapped display coordinates on the progress bar, the tag information can be overlaid on the progress bar at the corresponding position as an icon or other prominent visual element. This allows the user to quickly jump back to the corresponding target frame by clicking or hovering over the tag during subsequent browsing, achieving efficient recording and revisiting of keyframe content.

[0030] The technical solution of this disclosure performs fine-grained analysis of target video segments through content importance scoring, and allocates more playback time and a larger progress bar display space to the more important time intervals through time scaling and non-linear progress bar mapping, making it easier for users to visually discover and access key segments. Simultaneously, based on non-uniform mapping, it can accurately convert the user's operation position into a specific frame on the original timeline during progress bar interaction, achieving precise positioning at the single-frame level. Furthermore, by generating marker information for the target frames and displaying it intuitively on the progress bar, users can easily annotate and repeatedly access key scenes.

[0031] In some embodiments, determining the content importance score for each time interval includes: acquiring video frames of the target video segment and metadata or text descriptions associated with the target video segment; inputting the video frames and associated metadata or text descriptions into a first large model; and acquiring the content importance score for each time interval output by the first large model. Here, a time interval is a continuous segment of the target video segment on the timeline, containing one or more frames. The content importance score is determined by the first large model based on the corresponding time interval, and its determination is based on at least one of the following: content information of the video frames within the time interval; semantic relevance between the content information and the metadata or text description; and whether the time interval contains a preset key event type.

[0032] Here, a video frame refers to the smallest unit of visual information that constitutes a video sequence, i.e., a still image of the video at a specific moment. Each video frame has a timestamp and frame number, used to identify its precise position on the timeline, enabling the content analysis results to be accurately mapped to a specific time interval. Metadata refers to supplementary information related to the target video segment but not directly displayed on the screen. It describes the video's attributes and context, reflecting the theme and importance of the video content indirectly, with an emphasis on the structural attributes of the target video segment. Textual description information refers to the video-related text content that uses natural language to explain or reconstruct the target video segment, with a greater emphasis on semantic content.

[0033] In this embodiment, all video frames within the determined start and end times of the target video segment are first extracted from the original video stream. These frames are then organized chronologically, and the timestamp and frame number of each frame are recorded. Subsequently, metadata and text description information related to the video segment, such as title, tags, introduction, chapter description, subtitles, or speech-to-text, can be read from the database or content management system associated with the video segment. Next, video frames are aligned with subtitles or speech-to-text based on their timestamps. Text segments falling within the same time interval are associated with their corresponding frame sets, and general metadata is associated with all time intervals of the target video segment. This allows the video frames, metadata, and / or text descriptions to be constructed as the input structure for the model.

[0034] Here, the first major model refers to a large-scale artificial intelligence model used to analyze video frames and their associated text information, and output the importance scores of content in each time interval. It can be a basic model that supports visual and textual multimodal input or a large model that is fine-tuned specifically for video understanding tasks. It can comprehensively understand video images, audio, subtitle semantics, as well as metadata and text descriptions, thereby making semantic judgments on the importance of different time intervals.

[0035] In this embodiment, the video frame sequence corresponding to each time interval and its associated text information can be grouped and input into the first large model according to the input format of the first large model. For example, the frame images can be sent to the visual encoding module, and the subtitles, tags, and descriptive text can be sent to the text encoding module. Then, the visual features and text features are jointly modeled through a multimodal fusion module. Further, during the inference process, the first large model utilizes its internal deep neural network structure to extract content information such as scenes, characters, and actions from the video frames, and combines this with semantic cues in the metadata and text descriptions to form a comprehensive semantic understanding of each time interval.

[0036] In this embodiment, the first model is constructed or trained during the training phase to have the ability to evaluate the importance of time intervals. Therefore, during inference, it outputs a content importance score representing the importance of each input time interval. For example, the content importance score can be a weight value normalized to the [0,1] interval, or it can be a multi-level score. After obtaining the content importance score, it can be stored one-to-one with the corresponding time interval to form a mapping table between time interval identifiers and content importance scores. The scores can be linearly or non-linearly normalized as needed.

[0037] Specifically, the first major model internally considers at least one of the following information when calculating the content importance score: First, the content information of video frames within the time interval, i.e., by performing object detection, scene recognition, face recognition, action recognition, etc. on the frame images to determine whether the time period contains key figures, key scenes, or intense actions, thus assessing its importance from a visual perspective; Second, the semantic relevance between video content information and metadata or text descriptions, i.e., by calculating the semantic similarity or matching degree between the image or subtitle content within the time interval and the overall title, tags, summary, chapter description, and other texts, if the content of a certain time interval highly matches the video theme or the description of a key chapter, the importance score of that interval will be increased accordingly; Third, whether the time interval contains preset key event types, i.e., several key event categories are set internally or externally within the model, and when the model detects the presence of these key events in the time interval, it automatically increases their importance score. Through the joint modeling and weighted fusion of the above multi-source information, the first major model can output a content importance score for each time interval that is more in line with human subjective feelings and business needs.

[0038] Thus, by introducing the first major model capable of processing both visual and textual information, multimodal semantic understanding and importance assessment of target video segments are performed at the time interval level. Compared to traditional methods, this approach can simultaneously utilize three information sources: video frame content, structured metadata, and textual descriptions. This allows for more accurate identification of time intervals highly relevant to the video's theme, key events, or business objectives. Based on this, a content importance score is generated for each time interval. This provides high-quality foundational data for subsequent functions such as time-lapse playback, non-linear progress bar display, intelligent summary generation, automatic point marking, and extraction of highlight segments. It also enables more intelligent highlighting of key content and de-emphasis on secondary content, significantly improving user search efficiency and viewing experience in long or complex videos.

[0039] In some embodiments, the preset key event types include at least one of the following: a preset object is contained in the time interval; a scene transition occurs in the time interval; a preset action or behavior is contained in the time interval; a keyword appears in the audio information corresponding to the time interval; a preset graphical user interface element is contained in the time interval.

[0040] In this embodiment of the disclosure, events involving a preset specific object or person within a time interval can be triggered by the detection and recognition of the target and face. For example, matching can be performed based on a preset list of objects or people (such as a ball, scoreboard, formula board, speaker, or specific guest). When the detector identifies a category or identity from the list within the time interval (identity verification can be completed by combining face embedding and a feature library), and the confidence level and visible area threshold are reached, the time interval is marked as containing a key event. This type is applicable to scenarios such as the appearance of a key player in sports, the appearance of formulas or blackboard writing in a classroom, or the appearance of a specific speaker in a meeting.

[0041] In this embodiment, the event type of scene transition occurring within a time interval can be detected through shot boundary and scene transition detection. For example, indices such as histogram differences in color or texture between adjacent frames, edge change rates, and abrupt changes in optical flow distribution can be calculated to identify different forms of transition, such as instantaneous jumps and gradual changes. Specifically, to reduce false cuts caused by rapid movement, threshold adaptation can be performed by combining camera motion estimation with change patterns within the time window, and background noise or changes in music segments in the audio layer can be used as supplementary evidence. Furthermore, after a scene transition is detected, this time interval is treated as a structurally critical event, which can be used to locate chapter boundaries, shot resets, or content theme changes. Specifically, in instructional videos, slide transitions and whiteboard clearing can be considered scene transitions; in edited content, switching between different camera angles or story segments can be considered scene transitions.

[0042] In this embodiment, the time interval contains preset action or behavior event types, which can be determined by a spatiotemporal action recognition model to identify the dynamic patterns of consecutive frames within the time interval. For example, the preset action list may include actions such as scoring a goal, dunking, raising a hand, pointing to a whiteboard, clicking a button, dragging a window, starting a presentation, and the appearance of a result. Motion features and contextual clues such as object interaction and tool status can be extracted from short time segments using optical flow, keypoint skeletons, and gesture trajectories. Then, a temporal network or video transformer can classify the actions and output the time window and confidence score of the action occurrence. Specifically, for short or fine-grained actions such as mouse clicks, multimodal cross-validation can be performed using screen highlighting, pointer trajectory, and sound effects; for long actions such as event occurrences, start and end boundary detection can be used to output the duration. Specifically, once an action is triggered, it is determined that the time interval contains a preset action or behavior.

[0043] In this embodiment, the event type of keywords appearing in the audio information corresponding to the time interval can be determined based on audio transcription and keyword retrieval. For example, Automatic Speech Recognition (ASR) can first be used to perform precise time-aligned text transcription of the audio, and custom word lists and pronunciation dictionaries can be applied under multilingual or domain-specific terminology to improve the recognition rate. Subsequently, keyword and phrase matching, synonym expansion, and topic clustering can be performed, and the occurrence position, repetition frequency, and intonation emphasis can be weighted to form trigger events. In particular, to avoid mismatches, consistency verification can be performed by combining context windows and visual evidence.

[0044] Here, the pre-defined graphical user interface elements refer to visual components inherent in the video content itself, added by the video creator, and possessing specific functions or indicative meanings. These can be graphical tags added by the video creator during post-production editing to emphasize the content. For example, in a science video, when a key point is mentioned, an exploded graphic of "Key Point" pops up on the screen; in a product introduction video, when a selling point is mentioned, a customized tag introducing the selling point appears on the screen.

[0045] In this embodiment, the time interval includes preset event types of graphical user interface elements. For screen recordings or interface demonstration videos, standard components can be identified using User Interface (UI) element detection and Optical Character Recognition (OCR), combined with template matching or icon vector features to confirm specific controls and state changes. For example, when a preset graphical user interface element is detected, it is marked as a critical event. Specifically, signals such as mouse pointer hover and click highlighting, keyboard shortcut overlay hints, and changes in the border of focused controls can be used as trigger enhancements to precisely define the interaction moment.

[0046] In this way, incorporating multiple key event types into a unified detection and scoring framework can form a complementary evidence system across the visual, audio, and text channels, significantly improving the accuracy and robustness of important frame recognition. At the same time, the cross-validation of multi-dimensional features effectively reduces false positives and false negatives.

[0047] In some embodiments, non-uniform time scaling is implemented using frame repetition or frame interpolation algorithms.

[0048] Here, frame repetition refers to extending the display duration of a segment by copying certain frames and playing them repeatedly in the original order without changing the content of the original frames, thereby achieving the effect of deceleration or time stretching. Frame interpolation algorithms, on the other hand, synthesize entirely new intermediate frames between adjacent original frames based on the relationship between their motion and content. This refines the temporal sampling density and maintains smooth motion during deceleration, significantly reducing motion blur and jitter.

[0049] In this embodiment, when using frame repetition to achieve non-uniform time scaling, the target local playback scaling curve can first be obtained from frame-level importance and then time-smoothed to avoid frequent rate jumps. Subsequently, the segment can be divided into short windows or the target scaling coefficient can be calculated frame by frame. The non-integer repetition requirement can be transformed into an integer number of repetitions per frame through error accumulation and jitter control, ensuring that the overall duration is consistent with the target scaling and the visual rhythm is uniform. Furthermore, at the audio layer, time scaling algorithms such as temporal synchronous superposition or phase vocoders can be used to stretch the sound proportionally while maintaining the pitch, and at the same time maintaining the mapping with the original time, which facilitates accurate jumps of the progress bar and positioning function.

[0050] Similarly, when using frame interpolation algorithms to achieve non-uniform temporal scaling, the local magnification of the target can be obtained based on importance, and a continuous temporal remapping function can be constructed to subdivide the interval requiring deceleration into denser sampling points. Subsequently, for each target's intermediate time point, its two neighboring frames can be selected as references, bidirectional optical flow and confidence can be calculated, and forward and backward consistency checks can be performed. After generating an occlusion mask, time-weighted motion compensation distortion and fusion can be applied to the reference frames. If necessary, depth estimation or edge protection can be superimposed to reduce ghosting and deformation. In particular, when scene switching or significant occlusion is detected, it can be automatically downgraded to frame repetition or directly skip interpolation frames to avoid artifacts.

[0051] Thus, by performing non-uniform temporal stretching within important segments, the frame repetition scheme achieves rapid duration extension with extremely low computational cost, making it suitable for edge devices and real-time scenarios. The frame interpolation scheme, while maintaining motion continuity, significantly reduces stuttering and visual discomfort caused by deceleration, providing higher viewing quality. Both can be combined with frame-level importance-driven temporal remapping and progress bar spatial remapping to give key segments on the progress bar higher spatial and temporal resolution, making it easier for users to accurately stop at key frames when dragging and previewing.

[0052] In some embodiments, non-uniform time scaling processing includes: for adjacent time intervals where the content importance score is higher than a preset importance threshold and the motion complexity is higher than a preset complexity threshold, performing frame interpolation processing between the first and last video frames using a video frame interpolation algorithm based on optical flow; wherein, the motion complexity is determined by calculating the average optical flow vector amplitude between consecutive frames within the adjacent time interval.

[0053] Here, the video frame interpolation algorithm based on optical flow first estimates the pixel-level motion field between two frames, and then performs time-based compensation and fusion on the two frames based on this motion to synthesize the intermediate frame. Optical flow describes the inter-frame displacement vector of each pixel. With the help of this motion information, the algorithm can pull the two frames to the middle position at the target intermediate time and generate a continuous and natural transition.

[0054] In this embodiment, the content importance score and motion complexity of each time interval can be obtained first on the timeline of the target video segment. The motion complexity is obtained by estimating the optical flow of all consecutive frame pairs within the time interval, calculating the optical flow vector amplitude of each consecutive frame pair and averaging it, and then statistically analyzing the average optical flow vector amplitude over the entire time interval. Subsequently, the content importance score of each time interval can be compared with a preset importance threshold, retaining only those time intervals with scores higher than the preset importance threshold. Then, the motion complexity of these candidate time intervals is compared with a preset complexity threshold, filtering out adjacent time intervals that simultaneously satisfy both a content importance score higher than the importance threshold and a motion complexity higher than the complexity threshold. Specifically, for each set of filtered adjacent time intervals, the first and last frames of the entire adjacent time interval can be determined on the timeline. The first and last frames are used as boundary frames for frame interpolation, and the number of intermediate frames to be inserted and their corresponding time positions are calculated based on the desired time scaling ratio or target frame rate. Based on this, a video frame interpolation algorithm based on optical flow can be performed on the first and last frames. First, the optical flow field of each pixel or feature point between the first and last frames is estimated. Then, according to the time ratio of each intermediate moment between the first and last frames, the optical flow vector is interpolated to derive the pixel position and color value corresponding to each intermediate moment. The required intermediate interpolated frames are then synthesized one by one. Finally, these interpolated frames are embedded between the first and last video frames in chronological order, so that the adjacent time interval is extended in time, the frame rate is increased, and a non-uniform time stretching segment after frame interpolation enhancement is formed.

[0055] In this way, it is possible to accurately identify adjacent time intervals that are both semantically critical and visually dynamically complex, and apply high-quality frame interpolation based on optical flow only to these segments that are most sensitive to the user's viewing experience. This allows for targeted slowing down of the playback pace of key actions or important events while ensuring overall playback duration and controllable computing resources. Furthermore, compared to simple frame repetition or linear blending, using optical flow for frame interpolation can more realistically reproduce the motion trajectory and deformation of objects in space, achieving continuity, smoothness, and naturalness in slow motion or time stretching, avoiding noticeable stuttering, ghosting, or distortion. This significantly improves the user's perception of clarity and immersion in exciting segments and key plots, optimizing the overall subjective experience of non-uniform time-stretched playback.

[0056] In some embodiments, non-uniform time scaling processing includes: setting a target total scaling factor M for the target video segment, where M is a number greater than 0; assigning a scaling factor to each time interval based on the content importance score of each time interval, wherein the time interval with a higher content importance score has a larger scaling factor; and determining the processed duration of each time interval based on the scaling factor.

[0057] Here, the total target scaling factor refers to the overall scaling ratio of the target video segment on the timeline, which can be the multiple of the output duration relative to the original duration. In particular, on the premise of a fixed output frame rate, the total target scaling factor can also be the multiple of the total output frames relative to the original total frames.

[0058] In the embodiments of the present disclosure, first, the adjustment target of the overall playback duration of the target video segment can be determined according to service requirements or user settings. Exemplarily, the original total duration of the target video segment can be read, and the target total duration of the segment after scaling processing can be calculated. Further, the total target scaling factor M can be calculated according to the ratio of the original total duration to the target total duration, where M is a real number greater than 0. In particular, when M>1, it means that the overall playback is slowed down or extended; when 0<M<1, it means that the overall playback is accelerated or compressed; when M is close to 1, it means that the overall duration remains basically unchanged but non-uniform adjustments are made internally. Among them, the sum of the durations of each time interval after processing is equal to the product of the original total duration of the target video segment and the total target scaling factor M.

[0059] Here, the scaling coefficient refers to a non-negative weight assigned to each time interval according to its content importance, which can be used to determine how much time share the time interval should obtain in the overall scaling quota. The larger the coefficient, the more the time of the time interval will be magnified.

[0060] In the embodiments of the present disclosure, first, all time intervals can be obtained, and the content importance scores corresponding to each time interval can be read. Subsequently, the content importance scores of all time intervals can be normalized and mapped to a standardized interval convenient for allocating scaling coefficients to avoid extreme imbalance of scaling coefficients caused by differences in scoring scales. On this basis, according to the total target scaling factor M and the normalized importance scores, a scaling coefficient can be calculated for each time interval. Exemplarily, a linear or non-linear function can be used to map high-scoring intervals to coefficients greater than 1, map low-scoring intervals to coefficients less than 1, and perform overall normalization or proportional adjustment on the scaling coefficients of all intervals so that the sum of the durations of all time intervals after scaling meets or approaches the overall duration requirement defined by M, thereby realizing the reallocation of time resources.

[0061] In this embodiment, the original duration and the original number of frames within each time interval can be read first. The target duration after processing for each time interval is then calculated based on the scaling factor. Subsequently, the target number of frames that each time interval should contain after processing can be derived by combining the target frame rate of the video or a strategy that maintains the original frame rate. For example, while keeping the frame rate constant, the target number of frames is proportional to the target duration. After obtaining the target number of frames, a suitable frame-level adjustment algorithm can be selected based on the scaling factor and the relationship between the target number of frames and the original number of frames. For example, when the scaling factor is greater than 1 and the target number of frames is greater than the original number of frames, new frames can be added between adjacent original frames through frame insertion or frame repetition. Further, when the scaling factor is less than 1 and the target number of frames is less than the original number of frames, the number of frames can be reduced through frame extraction or frame merging algorithms, such as extracting frames at certain intervals or merging several adjacent frames in time to compress the playback duration of the interval. Finally, after adjusting the number of frames in each time interval, the processed frame sequence can be reassembled into a new target video segment, so that the actual playback duration of each time interval is consistent with the corresponding target duration, thereby completing the non-uniform time scaling processing based on the scaling factor.

[0062] In this way, while ensuring that the overall duration target is achieved, time resources are prioritized for video segments with more critical content, allowing important details to be displayed for a longer period and with higher temporal resolution, while secondary content is appropriately compressed, thereby improving users' perception and understanding of key events.

[0063] In some embodiments, setting a target total scaling factor for a target video segment includes: setting the target total scaling factor based on the original duration of the target video segment.

[0064] Here, the original duration refers to the playback length of the target video segment on the timeline before any acceleration, deceleration, frame interpolation, or cropping / recalculation. It can be based on the media's timestamp and accurately covers the actual presentation time between the segment's in-point and out-point, unaffected by the encoded frame type, decoding buffer, or playback device speed settings.

[0065] In this embodiment, the original duration of the target video segment can be determined first based on the presentation timestamp. Then, the overall scaling ratio of the segment can be determined by considering the overall duration budget, platform or business duration limitations, and narrative and interaction requirements. For example, segments with excessively short original durations tend to be significantly enlarged to achieve the minimum perceptible display time, while segments with excessively long original durations are moderately compressed to improve the viewing pace.

[0066] Thus, by using the original duration as an objective baseline and establishing the scaling factor on measurable and reproducible temporal facts, the magnification or compression of different segments not only conforms to their own length characteristics but also meets the constraints of overall duration and visual perception, thereby avoiding the imbalance caused by subjective, arbitrary scaling factor selection. Through constraint and smoothing mechanisms, visual discomfort and auditory disjointness caused by sudden speed changes are reduced, and a clear target and stable time mapping are provided for subsequent frame insertion or frame repetition, ultimately achieving a more predictable final cut, a more coherent viewing experience, and higher production efficiency.

[0067] In some embodiments, providing frame-level positioning based on a progress bar includes: providing a forward jump button and a backward jump button on the video player interface; in response to triggering any button, controlling the video playback to jump forward or backward by a predetermined number of frames; wherein the predetermined number of frames are video frames processed by non-uniform time scaling, and the value of the predetermined number is adjusted according to the content importance score distribution of the time interval containing the current playback position and its adjacent time intervals.

[0068] Here, the video player interface refers to a visual operating interface used to present video content and provide interactive control. In this embodiment of the disclosure, the video player interface may include elements such as play / pause, progress bar, time indicator, and subtitle switch, which can serve as the entry point for users to watch and locate content.

[0069] Here, the "Jump Forward" button refers to an interactive control in the interface used to move the playback position forward along the timeline (i.e., towards the past). A single trigger moves the playback point forward several frames, and it can be represented as an icon button, keyboard shortcut, or gesture. The "Jump Backward" button refers to another interactive control in the interface used to move the playback position backward along the timeline (i.e., towards the future). A single trigger moves the playback point backward several frames, and it can also be implemented through a button, keyboard shortcut, or gesture.

[0070] In this embodiment, two clickable controls can be designed and placed in the player's control layer, clearly defining their semantics and direction, and providing equivalent entry points for touch, mouse, and keyboard on different terminals. Specifically, frame-level positioning commands can be bound to the buttons instead of just coarse-grained jumps based on time. The commands internally ensure frame accuracy under variable frame rates and compressed encoding conditions through a bidirectional mapping between timestamps and frame indices. Furthermore, state management and reachability design can be provided for the buttons, including disabled or weakened display at boundaries, prompts for the current frame number and remaining jumpable range, screen reader labels, and high-contrast adaptation. User-triggered events are recorded in the console for subsequent optimization.

[0071] In this embodiment, when any jump button is triggered, a local time window can be extracted centered on the current playback position. The importance scores of the content in each time interval within the window are read to form a distribution feature, and the specific value of the predetermined number of jumps is determined accordingly. That is, the predetermined number of jumps is a dynamically adjustable value. For example, a smaller step size is used in areas with dense importance or drastic changes to avoid skipping key details, while a larger step size is used in sparse or lengthy segments to improve browsing efficiency. Specifically, minimum and maximum step sizes, scene switching protection, and decoding accessibility constraints can be applied simultaneously to ensure that the jumps are both smooth and feasible. Subsequently, forward or backward movement can be selected according to directional instructions to locate the target frame and handle addressing across keyframes and buffer boundaries, completing decoding warm-up and scene switching, while updating the progress bar, highlighting the current frame, and other interface states. Specifically, audio and subtitle tracks can be synchronized for a consistent overall experience.

[0072] Thus, by providing explicit bidirectional frame-level navigation at the interface layer and automatically adapting the step size to the importance of local content at the execution layer, users can achieve more refined frame-by-frame browsing in information-dense areas and more efficient cross-frame scanning in information-sparse areas. Accurate positioning and boundary protection reduce navigation errors and image jitter caused by variable frame rates, encoding structures, or scene switching. Unified audio, video, and subtitle synchronization maintains perceptual consistency, improving the navigation and interaction efficiency of long videos.

[0073] In some embodiments, the predetermined quantity corresponding to the time interval with high content importance score is less than the predetermined quantity corresponding to the time interval with low content importance score.

[0074] In this embodiment of the disclosure, when a user triggers a jump, a local time window can first be established with the current playback position as the center. The representative importance level of the window is obtained by comprehensively considering signals such as motion intensity, salience, character and text detection, voice events and camera boundaries. Then, based on the measurement that the higher the importance, the smaller the step size, and the lower the importance, the larger the step size, the number of frames for this jump is determined by step size comparison or rule mapping, so that the predetermined number corresponding to the area with high content importance score is less than the predetermined number corresponding to the area with low content importance score.

[0075] In this way, users can obtain a detailed frame-by-frame browsing capability in information-dense areas, avoiding skipping key details, while reducing unnecessary stops in information-sparse areas, thus improving overall navigation speed and viewing efficiency.

[0076] In some embodiments, generating tag information to identify the location corresponding to the target frame includes: performing a hash operation on at least one of the timestamp information of the target frame, the unique identifier information of the video, and the user identifier to generate tag information.

[0077] Here, timestamp information refers to the precise location data of the target frame on the timeline, which can be a Presentation Time Stamp (PTS) that uniquely identifies the playback time of the frame in the video. Unique identifier information refers to an identifier that can stably identify a segment of video material, which can include a global identity document (ID) from the content library, media fingerprints, content hashes, etc. User identifiers refer to stable identifiers associated with end users or accounts, which can be user IDs, device IDs, or alternative identifiers. Hash operations refer to an irreversible mathematical transformation that maps an input of arbitrary length to a fixed-length digest. Collision-resistant, pre-mapping-resistant cryptographic hashes or keyed message authentication codes can be used to obtain stable, compact, and difficult-to-forge token values.

[0078] In this embodiment, the presentation timestamp of the target frame, the unique video identifier, or the de-identified user identifier can be selected as hash participation fields according to the intended use. The timestamps are unified to a single time base, and all fields are normalized for case sensitivity, whitespace, and Unicode. Subsequently, the fields can be serialized into an unambiguous byte stream according to a predefined order. Specifically, in scenarios involving user dimensions or requiring anti-counterfeiting measures, a keyed message authentication code or the addition of a random salt can be used to resist enumeration and forgery. Finally, after completing the hash calculation, it can be truncated to a fixed length as needed, thus forming a compact tag string. In particular, this tag information can be verified by recalculating according to the same rules during retrieval or sharing, ensuring uniqueness, stability, and difficulty in tampering.

[0079] Thus, by combining keyed hashing to achieve a balance between determinism and anti-counterfeiting of tag information, tags generated for the same target frame across different devices and times can be stably aligned, while effectively preventing enumeration and forgery even when user dimensions are included. Through compact, fixed-length digests, tags can be efficiently transmitted and indexed in URLs, logs, and databases, significantly reducing the risk of collisions and tampering.

[0080] In some embodiments, the interaction method of the video progress bar further includes uploading the marker information and its associated target frame position information to a server for storage.

[0081] In this embodiment, the client first organizes the unique video ID, the target frame timestamp unified to a fixed time base, the tag value, and necessary version metadata into a deterministically serialized data packet, adhering to privacy minimization and anonymizing user-related fields. Subsequently, a short-term access token is obtained and integrity protection is added to the request body. The upload interface is called via encrypted communication, and timeout and retry keys are set to prevent repeated writing under weak network conditions. Conversely, the server entry layer completes authentication and anti-replay, while the application layer verifies the field format, video ID validity, and timestamp range, and recalculates the tag according to the same rules to prevent tampering. Upon successful verification, the record is written to encrypted storage and an index is created. Finally, the server returns confirmation information, which the client uses to update its local cache and status.

[0082] In this way, by uploading and centrally storing the tagging information and its target frame location information on the server side, stable positioning and sharing capabilities are achieved. Deterministic serialization and verification mechanisms ensure that the same content generates consistent tags, reducing the risk of collisions and tampering. Efficient indexing and persistence make retrieval and navigation fast and reproducible. Combined with auditing and backup, this improves operational and compliance capabilities, ultimately significantly enhancing stability, data reliability, and overall interaction efficiency under a scalable backend architecture.

[0083] In some embodiments, the interaction method of the video progress bar further includes: in response to the target video segment being reloaded and played, obtaining all publicly available tag information related to the target video segment from the server; classifying the tag information into a first type of tag created by the current user and a second type of tag created and publicly available by other users; and outputting the first type of tag and the second type of tag at the corresponding position of the progress bar; wherein the visual styles of the first type of tag and the second type of tag are different.

[0084] In this embodiment of the disclosure, when a target video segment is reloaded and started playing by the current user, a request can first be sent to the server based on the segment's unique identifier (such as video ID and segment time range). After receiving the request, the server retrieves all tag information set to "publicly visible" from the tag database associated with the target video segment and packages and returns this tag data. Subsequently, after receiving the data returned by the server, it can be used as the basic data set for subsequent tag classification and display.

[0085] Here, the first type of marker refers to the marker information created by the user currently watching the video, representing the time points or frame positions that the user personally considers important. The second type of marker refers to the marker information created by other users and chosen to be public. The server stores and manages these public second-type markers. When any user reloads the same target video segment, these markers created by others but set to public will also be retrieved and displayed for reference and sharing.

[0086] In this embodiment, the user ID of the currently logged-in user can be compared with the creator user ID in each tag record. Tags whose creator ID matches the current user ID are classified into the first category of tags, while tags whose creator ID does not match the current user ID and whose public attribute is "public" are classified into the second category of tags. These tags are then stored in their respective tag lists or data structures. Subsequently, different visual styles can be assigned to the two categories of tags according to preset UI design rules, thereby completing the source classification at the data level and preseting different display styles at the presentation level.

[0087] In this embodiment of the disclosure, the timestamp or frame number associated with each marker can be read, converted into a relative position within the overall playback duration, mapped to pixel coordinates on the progress bar, and the marker graphic can be drawn at the corresponding position. For example, for the first type of marker, a corresponding visual style can be invoked for rendering; for the second type of marker, another visual style that is clearly distinguishable from it is invoked.

[0088] Thus, the first type of marker is highlighted with a unique visual style, making it easy for users to quickly locate and access key segments in their historical records, improving the continuity and efficiency of personal learning, review, or editing. The second type of marker is presented with a different visual style, allowing users to intuitively utilize the experience and annotation results accumulated by others to quickly discover key content or high-value segments in the video, thereby reducing repetitive work and improving the overall interactive value and user experience of the video progress bar.

[0089] In some embodiments, the interaction method of the video progress bar further includes: receiving a user's trigger operation on any marker information on the progress bar; in response to the trigger operation, obtaining and displaying details related to the marker information, the details including at least one of creator information, annotation text, associated keyframe thumbnails, or a shortcut button to jump to the corresponding position of the marker information.

[0090] In this embodiment of the disclosure, when rendering the progress bar markers, each marker can be pre-drawn as a focusable and clickable interactive element, with a clear hit area and accessibility attributes set, and mouse, touch, and keyboard event handlers registered uniformly. Furthermore, when a user clicks, taps, or confirms with the keyboard, hit detection and debouncing can be performed first to distinguish between the intention of dragging the progress bar and clicking the marker, and then the ID and timestamp of the currently selected marker can be locked.

[0091] Here, "details" refers to the structured set of information associated with the tag, used to help users understand its meaning and context. Specifically, "creator information" refers to displayable metadata related to the source of the tag information, such as display name, avatar, authentication status, and privacy-protected identifiers; "annotation text" refers to the creator's written description of the time point or segment, which may include a brief title and body text; "associated keyframe thumbnails" are static preview images taken from keyframes near the tagged time point, used to quickly convey the content of the scene; and "shortcut buttons" are operable interface controls that, when clicked by the user, allow the player to quickly locate the time position corresponding to the tag and begin playback.

[0092] In this embodiment, in response to a trigger operation, based on the currently selected marker ID and timestamp, the details container (such as a bubble card or side panel) anchored to the marker can be opened to display a skeleton screen or placeholder information. Specifically, the client calls the marker details interface using the marker ID and video ID as query keys, requesting fields including a list of required fields, time base version, etc. Subsequently, the server returns validated structured data, which may include creator information, annotation text, keyframe thumbnail references and their signature URLs, marker normalized timestamps, etc. Next, after the client completes pattern validation and time base alignment, the annotation text is rendered securely, and the thumbnails are lazy-loaded and degraded placeholders are used. In particular, the details panel can provide a prominent shortcut button whose event is bound to the player interface. Clicking it immediately positions the playback pointer to the marker time point and provides visual or auditory feedback, while simultaneously updating the progress bar highlight and browsing history.

[0093] In this way, the interaction flow achieves low-latency, accessible, and secure disclosure of details in both the triggering and display phases. Creator information and annotation text enhance the interpretability and credibility of the markers, while thumbnails significantly improve users' instantaneous recognition of the scene and decision-making efficiency, thereby improving the overall user perception, navigation, and interaction quality of the progress bar markers.

[0094] In some embodiments, the interaction method of the video progress bar further includes: performing differential rendering of at least one of color, texture, or transparency on the portion of the remapped progress bar corresponding to the target video segment, based on the content importance score distribution of the target video segment.

[0095] In this embodiment, the importance scores of each time segment within the fragment can first be normalized to [0,1] using a unified time base, and the fragment can be mapped to pixel intervals on the adjusted progress bar. Subsequently, the fragment can be divided into several segments using an equal-length or adaptive strategy, and adjacent scores can be smoothed to avoid abrupt changes. Specifically, during rendering, the importance of each segment can be mapped to the visual channel. For example, for color, a perceptibly uniform, colorblind-safe color band can be used with a gradient from low to high (e.g., cool to warm or light to dark); for texture, tiling stripes or dot patterns can be used, with density increasing or direction changing as importance increases; for transparency, it can be set to become more opaque as importance increases. Finally, gradient fill, texture tiling, or semi-transparent mask overlays can be applied within the segment range of the progress bar, and interpolation transitions can be applied to the segment boundaries to ensure a consistent visual experience.

[0096] In some implementations, for online education platforms, specific segments are those explaining core knowledge points in course videos, and the tagging information is used to identify teaching focus or difficulties. For sports event video analysis, specific segments are those containing key attacks or scores, and the tagging information is used to identify highlights. For product introduction videos on e-commerce platforms, specific segments are those showcasing the core functions or features of the product, and the tagging information is used to identify the product's selling points.

[0097] In this way, by visually presenting the distribution of the importance of different sections of content within a segment on the adjusted progress bar using color, texture, or transparency, users can quickly identify key and non-key areas, thereby optimizing browsing and navigation decisions and reducing blind dragging and repeated viewing.

[0098] In some embodiments, the segment in the progress bar corresponding to the target video segment is remapped and displayed according to the time ratio after scaling, including: calculating the ratio of the total duration of the target video segment after scaling to the original duration of the target video segment; and stretching the segment corresponding to the target video segment on the progress bar according to the ratio, while keeping the display ratio of the rest of the progress bar unchanged.

[0099] Here, total duration refers to the actual playback duration of the stretched video segment in the current version. In this embodiment of the disclosure, total duration is the new duration formed after the video segment has undergone speed-changing processing, editing and rearranging, frame interpolation and extension, or generation and replacement, and can be measured using a unified time base.

[0100] In this embodiment of the disclosure, when a user selects or enters a specific segment, the client first determines the original time range of the segment and calculates the original duration; then it reads the time-series metadata of the scaled version corresponding to the segment, obtains the new duration on the same time base, calculates the ratio, and records the version and source of the ratio for use in subsequent rendering and interactive mapping.

[0101] Here, stretching the display refers to expanding or shrinking the display width corresponding to the segment in a proportional manner at the visual level of the progress bar, so that the pixel length occupied by the segment on the progress bar is proportional to its stretched duration, while the pixel-time ratio of the remaining time intervals of the progress bar remains unchanged, thus forming a non-linear time axis with different segment ratios.

[0102] In this embodiment, a piecewise linear mapping function can be constructed for the progress bar, remaining unchanged outside specific segments while adopting a scaled-up or scaled-down approach within the segments. Specifically, the segment's interior can be mapped proportionally, while the areas before and after the segment remain unchanged, thus visually stretching or compressing the segment proportionally, while the proportions of other segments remain constant. Subsequently, the rendering layer can rearrange the scales, segment backgrounds, and marker positions on the progress bar accordingly, update the segment's display width and gradient / texture / transparency bands, and use smooth transition animations to avoid abrupt changes. Next, the interaction layer can simultaneously implement the inverse mapping of clicks and drags, ensuring accurate positioning of the user within the stretched segments, alignment of thumbnails and keyframe previews, and enabling horizontal scrolling or adaptive scaling to accommodate the expanded area when the overall width of the progress bar increases, without altering the proportions and hit logic outside the segments.

[0103] In this way, by stretching the clips on the progress bar according to the actual playback duration, while keeping the proportions of other segments unchanged, users can intuitively perceive the change in duration and narrative weight of the clip relative to the original clip, thus improving navigation efficiency.

[0104] In some embodiments, a frame-level positioning operation is provided based on a progress bar, including: providing a keyframe preview mode as the entry point for the frame-level positioning operation; in the keyframe preview mode: extracting frames of the target video segment; inputting the extracted frames into a second large model and obtaining the independent importance score for each frame output by the second large model; based on the independent importance score, selecting keyframes from the target video segment to form a keyframe set; displaying the keyframe set in the form of an image wall in the user interface; and responding to the selection operation of the target keyframe in the image wall, controlling the video player to jump to the first time point in the target video segment corresponding to the target keyframe.

[0105] In this embodiment of the disclosure, an entry control for a "keyframe preview mode" can be preset in the video progress bar or playback interface. For example, a button or icon can be placed near the progress bar, or the mode can be triggered by long-pressing the progress bar or double-tapping a specific area. When the user clicks the button or triggers a specified gesture, the user switches to the keyframe preview mode interface. For example, the playback of the current video can be temporarily paused or minimized to one side, while reserving an area on the interface for a subsequent image wall display of keyframes.

[0106] In this embodiment of the disclosure, after entering the keyframe preview mode, the original encoded data of the video segment can be read first, and the image frames can be decoded frame by frame by the decoder. The timestamp and frame number of each frame are recorded in chronological order. In particular, in order to balance efficiency and representativeness, full-frame extraction or sampling extraction at fixed intervals can be adopted, such as retaining one frame every several frames or several milliseconds, while maintaining the mapping relationship between each frame and its time position in the original video.

[0107] Here, the second major model refers to a large-scale artificial intelligence model used for understanding and assessing the importance of single-frame images. This model can be a visual model trained on a large amount of image or video frame data, based on deep neural networks. Compared to the first major model, which assesses content importance over time intervals, the second major model focuses more on analyzing each frame itself. This includes identifying people, objects, scenes, text, or important actions within the frame and assigning a score based on this information to determine the frame's representativeness and reference value within the entire video. The independent importance score is a numerical value assigned by the second major model to each frame individually, independent of its preceding and following frames. The second major model treats each frame as an independent image sample, analyzes it separately, and outputs a score reflecting the frame's information content, representativeness, or visual and semantic saliency. Specifically, the independent importance score can be used to measure whether a frame is suitable as a representative image for a user's quick browsing.

[0108] In this embodiment, after necessary image preprocessing of the extracted frame images, these preprocessed frames can be input into a second large model in a certain batch. Subsequently, the second large model can perform visual feature extraction and semantic analysis on each frame individually, such as identifying whether key figures, significant actions, complex scenes, important objects, or text information appear in the image, and output an independent importance score for each frame based on the decision rules within the model or the patterns learned during the training phase. In particular, after the second large model completes inference, the scores corresponding to all frames can be collected and stored together with the frame timestamps and frame sequences to form a mapping dataset.

[0109] Here, a keyframe refers to a small subset of representative frames selected from all frames in a video. These frames effectively summarize a particular segment of content or contain important visual information. Compared to regular frames, keyframes allow users to quickly grasp the main structure and key points of the video.

[0110] In this embodiment, an importance threshold can be set, or a certain percentage of high-scoring frames can be selected. Frames with scores higher than the threshold or ranking high are considered candidate keyframes. Specifically, to avoid keyframes being too concentrated in time, time interval constraints or clustering algorithms can be introduced to select frames with more even time distribution and more significant content differences from the candidate frames as the final keyframes. For example, adjacent high-scoring frames can be merged, retaining the frame with the highest score. Finally, a set of filtered keyframes is obtained, and the thumbnail data and corresponding time point information are retained for each keyframe.

[0111] Here, "image wall" refers to a display method in which multiple image thumbnails are presented in a grid format within a user interface. In this embodiment of the disclosure, the image wall can contain multiple images arranged in rows and columns in a compact manner, and multiple keyframe thumbnails can be displayed on the same screen.

[0112] In this embodiment, the width and height of the image wall area can be predetermined. The thumbnail size and number of rows and columns are calculated based on the screen size and the desired number of rows and columns. The keyframe thumbnails are then placed sequentially in row-first or column-first order, and each thumbnail is appended with its corresponding time point or brief prompt information. For example, a timecode (such as "00:12:35") or a short text label can be displayed below the thumbnail. Simultaneously, click or touch event listeners can be bound to these thumbnails, allowing users to scroll through and select any keyframe in the image wall.

[0113] In this embodiment, when a user clicks or touches a target keyframe thumbnail in the image wall, the interaction event can be captured, and the corresponding time point recorded in the data structure for that keyframe (i.e., the first time point of that frame in the target video segment, such as the timestamp of that frame or the keyframe time position of that frame) can be read. Then, a jump command is sent to the video player to precisely position the playback progress to that time point. After receiving the jump command, the player adjusts the playback position to the target time point and decides, according to the configuration, whether to immediately resume playback or pause at that frame and wait for user confirmation. In particular, the keyframe preview mode interface can also be automatically closed or collapsed, returning the interface focus to the video playback window, allowing the user to seamlessly continue watching the corresponding segment after selecting a keyframe, thereby completing a frame-level positioning operation based on keyframes.

[0114] Thus, a keyframe preview mode based on a second major model, automatically selecting keyframes, is added to the traditional progress bar. This allows users to quickly browse representative frames selected by the model through the image wall and jump to the corresponding position with one click. The second major model provides an independent importance score for each frame, ensuring that the selected keyframes are mostly more informative and representative, making the image wall a concise and efficient summary of the video's main content structure. The image wall's visualization is intuitive and easy to understand. Users can form an overall understanding of the video's content distribution by browsing a few thumbnails and quickly locate the segments they are interested in. This significantly reduces the time cost and operational burden of finding specific content in long or complex videos, improves the accuracy of frame-level positioning and the interactive experience, and has significant practical value for learning, reviewing, and debriefing scenarios.

[0115] In some embodiments, after jumping to the first time point, the interaction method of the video progress bar further includes: providing a frame-level positioning control; in response to the operation of the frame-level positioning control, performing frame-level precision forward or backward fine-tuning from the first time point; determining the final frame located by the fine-tuning operation; wherein the final frame is the target frame.

[0116] Here, frame-level positioning control refers to an interactive control in the video player interface used to finely adjust the playback position in single-frame units. It allows users to move the playback position forward or backward frame by frame by clicking buttons, key presses, or fine-tuning sliders after they have roughly jumped to a certain point in time, thereby achieving precise positioning operations down to the specific frame.

[0117] In this embodiment of the disclosure, after the video player jumps to the first time point, frame-level positioning controls can be displayed or activated near the player according to a preset interface layout and interaction rules. For example, "previous frame" and "next frame" buttons can be presented below the timeline or next to the video frame, or frame advance and frame retreat operations can be bound to keyboard shortcuts, or fine-grained adjustment sliders can be provided. These controls are then associated with the frame indexing and decoding modules within the player, thereby providing an entry point for users to perform frame-level fine-tuning near the current time point.

[0118] In this embodiment, after detecting a user's specific operation on the frame-level positioning control, the frame index can be increased or decreased according to a preset stepping rule, based on the current frame at the first time point. For example, each operation increments or decrements the current frame number by 1, or supports continuous multi-frame stepping when needed. Subsequently, the player decodes the corresponding target frame from the cache or video stream based on the updated frame number and immediately refreshes the video display, allowing the user to see the latest image after fine-tuning forward or backward by one frame in real time, thereby perceiving the impact of each small adjustment on the positioning result. Specifically, when the user continuously operates the frame-level positioning control, the playback position can be continuously updated frame by frame, ensuring that the fine-tuning process maintains frame-level accuracy at all times.

[0119] In this embodiment of the disclosure, after the user completes the frame-level fine-tuning operation, the frame currently where the player is positioned is determined to be the final frame of this positioning process. Subsequently, the frame number and timestamp corresponding to the final frame can be recorded as the output result of this frame-level positioning, and defined as the target frame for subsequent functions such as marking, screenshotting, analysis, or continuing playback.

[0120] This further provides a fine-tuning mechanism with frames as the smallest unit, allowing users to quickly and accurately adjust the playback position to a specific target frame without repeatedly dragging the progress bar or testing playback, significantly reducing positioning errors. The introduction of frame-level positioning controls elevates video browsing to "frame-level" precision, making it particularly suitable for scenarios requiring precise pausing at a specific moment of action, a point of change in the frame, or the appearance of a subtitle. This improves the accuracy and efficiency of operations such as marking, editing, evidence collection, and teaching, and significantly optimizes the interactivity of the video progress bar and the user experience.

[0121] In some embodiments, displaying a set of keyframes in the form of an image wall in the user interface includes: establishing a mapping relationship between the independent importance score of each keyframe in the keyframe set and its display size; determining the display size of each keyframe based on the mapping relationship and displaying it, wherein the keyframe with the higher the independent importance score has a larger display size.

[0122] Here, display size refers to the size of each keyframe thumbnail displayed on the screen in the image wall of the user interface, including its display width and height. That is, the size of the visible area occupied by the same original keyframe image after scaling without changing its content composition.

[0123] In this embodiment, the independent importance score corresponding to each keyframe in the keyframe set can be read first, and all scores can be statistically analyzed and normalized. Then, based on interface design requirements and screen resolution, the minimum and maximum allowable display sizes of the keyframe thumbnails in the image wall can be pre-defined. Next, a functional relationship is defined to map the standardized importance scores to the display sizes. This mapping can be a linear or non-linear function, and the mapping can be appropriately compressed or stretched according to the image wall layout.

[0124] In this embodiment, for each keyframe in the keyframe set, its independent importance score can be read first, and then substituted into the established mapping function to calculate the corresponding target display size. Specifically, based on the actual width, height, and number of thumbnails that the image wall area can accommodate, necessary overall fine-tuning or adaptive layout processing can be performed on the target size of all keyframes to avoid layout overflow or over-density. Subsequently, according to the calculated size, each keyframe image can be proportionally scaled to generate thumbnails. Thumbnails of different sizes are arranged in the image wall according to a predetermined layout strategy (such as waterfall layout, irregular grid layout, or layout based on prioritizing large images), ensuring that keyframes with higher independent importance scores are presented with a larger display size in the image wall, making them more visually prominent and eye-catching.

[0125] In this way, a direct link is established between the independent importance score of a keyframe and its display size in the image wall. Within the same interface, the visual cue of size difference highlights keyframes with higher semantic or informational content. This allows users to focus their attention on more important key images in a very short time when browsing the image wall, without having to read text descriptions or examine every thumbnail. This improves the information carrying and retrieval efficiency of the keyframe preview interface, making it easier for users to discover and select truly valuable frames for navigation or analysis, reducing attention and clicks on unimportant images. By reasonably controlling the minimum and maximum display sizes and combining them with adaptive layout, the overall interface's aesthetics and readability are maintained while increasing the hierarchy and information expression of the image wall display, thus significantly optimizing the user experience during keyframe preview and frame-level positioning.

[0126] In some implementations, the tagging information and corresponding video clips can be sent to a large model used to generate the video. The large model can then generate a better new video based on the tagging information. For example, the client can first integrate the timestamp, type, importance, annotation text, and keyframe thumbnails of each tag according to a unified time base, and then crop the corresponding video clips around each tag, transcoding the clips to an encoding format, resolution, and frame rate that the large model can interpret. Subsequently, a structured description containing the tag list can be constructed and submitted to the video generation large model along with text prompts and reference keyframes, carrying compliance and privacy policy annotations. Further, the video generation large model uses the tags as temporal constraints and narrative anchors during the generation phase. For example, it can adjust shot duration and attention based on importance weights, use annotation text to drive on-screen content and narration, use keyframes as conditional inputs for on-screen style or composition, and generate or rearrange shots according to the tag order to achieve a new version of the video that emphasizes key points and optimizes pacing and transitions. Finally, after generation, the video generation large model can return results and alignment information, which the client can use to verify the temporal sequence and archive it to support subsequent iterations and feedback loops.

[0127] The technical solution disclosed herein enables non-uniform time scaling of specific segments based on content importance, giving physically equal progress bar intervals differentiated time accuracy. Key areas are magnified, significantly improving drag-and-position accuracy and solving the problem of traditional linear progress bars struggling to accurately locate critical content. It organically combines coarse adjustments on the scaled progress bar with frame-level fine adjustments, allowing users to quickly approach the target before precisely locking on, significantly reducing the number of operations and positioning time, achieving both high efficiency and accuracy. By generating and associating unique marker information with the target frame, assigning addressable digital coordinates to each frame, it supports personal recording and playback, enhancing the discoverability and interactivity of video content.

[0128] In some implementations, importance-based non-uniform frame padding and timeline remapping can be performed on a user-selected time interval. Specifically, the importance of each frame within the interval can first be evaluated, with evaluation sources including preset rules (such as motion amplitude, detected faces / text / events) and large model-assisted scoring, resulting in an importance score of [0,1]. Then, a target duration expansion factor K is determined (e.g., expanding 1 second to 10 seconds), and non-uniform time expansion weights are assigned to each frame based on its importance: high-importance frames can be assigned a higher magnification factor (e.g., 20 times), and low-importance frames are assigned a lower factor (e.g., 5 times). Normalization ensures that the total duration after expansion is approximately equal to K times the original duration. The rendering and playback layers then perform frame-level time resampling / repetition (or frame interpolation) to generate a "scaled" output sequence; simultaneously, the visual width of the corresponding segment of the progress bar is proportionally enlarged by the overall expansion factor K, forming a non-linear timeline, but the basic drag step / playback interval of the progress bar maintains its original rhythm (e.g., still in 1-second steps). From the user's perspective, it's like the key areas are intelligently "slowed down," making it easier to accurately select the target content when dragging and previewing.

[0129] Furthermore, to improve positioning efficiency, two types of fine-tuning interactions can be provided: First, a step-by-step jump operation. After pausing at any location, the user can use the "forward / backward" buttons to perform discrete micro-step jumps within the frame neighborhood, jumping n frames at a time and automatically stopping. n can be dynamically optimized based on user habits, content complexity, and accidental touch rate (e.g., adaptively switching between 1, 3, and 5 frames). Second, a unique location marker. A persistently addressable unique ID can be generated at each confirmed target location and bound to the location's time mapping, keyframe thumbnail, and contextual metadata. Users can directly and quickly jump to the same semantic location using this ID, even if the underlying timeline changes due to further adjustments.

[0130] Furthermore, to enhance collaboration and discovery, users' own marker positions can be overlaid on the progress bar. With authorization, shared markers and highlights from other users can be aggregated and visualized as dots, logos, or heat bars. Hovering over these markers allows users to view descriptions and importance weights, helping them quickly identify keyframes or segments of collective interest within the zoomed-in area. This achieves precise two-step positioning—coarse and fine—for non-uniform temporal zooming of important content, without altering the user's familiar drag-and-drop and playback interval perception.

[0131] In some implementations, the video can first be processed frame-by-frame and visually segmented according to the user-selected time interval. For example, a 2-second segment can be extracted at 24 frames per second, resulting in 48 frames, and corresponding frame images and an image wall can be generated. Further, to reduce the difficulty of subsequent filtering, redundant frames can be discarded based on a content difference threshold between adjacent frames, retaining only representative frames with significant changes. Simultaneously, a large model can be integrated to perform semantic judgment and importance assessment of frame content, automatically labeling possible final frames or regions as a priority candidate set for user filtering, thereby reducing the burden of manual frame-by-frame comparison. Furthermore, at the interaction level, the player supports pause and step-by-step navigation strategies. Specifically, when the user clicks pause at any position, the interface provides two step buttons: forward and backward. Each click will jump n frames on the timeline according to a configurable step size n and automatically stop, facilitating quick location. The step size n can be dynamically optimized based on content importance, adaptive rules, or user preferences (e.g., decreasing n for high-importance intervals and increasing n for low-importance intervals) to balance positioning accuracy and browsing efficiency. When a user identifies a location as a key point, a unique identifier can be generated for that location, along with its timestamp, associated frame, and necessary semantic tags. Specifically, this identifier is displayed as dots on the image wall, supporting hovering to view details and one-click navigation back. To promote collaboration and interaction, other users' public location information and marked dots can be displayed simultaneously on the image wall, supporting filtering, access control, and facilitating cross-user reference, discussion, and review.

[0132] Through the above process, a highly efficient video positioning experience is achieved by combining frame-by-frame visual segmentation, intelligent candidate localization, and auditable step navigation.

[0133] This disclosure provides an interactive device for a video progress bar, such as... Figure 2 As shown, the device may include: a scoring determination module 201, used to divide the target video segment into multiple time intervals in response to a selection instruction for the target video segment, and determine the content importance score for each time interval; a video scaling module 202, used to perform non-uniform time scaling processing on the target video segment based on the content importance scores of each time interval, wherein the time interval with a higher content importance score is allocated a longer playback duration; a scaling adjustment module 203, used to remap and display the segment in the progress bar corresponding to the target video segment according to the time ratio after scaling processing; a positioning provision module 204, used to provide frame-level positioning operation based on the progress bar; and a marker generation module 205, used to generate marker information indicating the position of the target frame in response to a marker instruction triggered when the positioning of a target frame is completed through frame-level positioning operation.

[0134] In some embodiments, the attribute determination module 201 includes: a data acquisition submodule, used to acquire video frames of the target video segment and metadata or text descriptions related to the target video segment; a model input submodule, used to input the video frames and associated metadata or text descriptions into a first large model; and a model output submodule, used to acquire the content importance score for each time interval output by the first large model; wherein, a time interval is a continuous segment of the target video segment on the timeline, containing one or more frames; the content importance score is determined by the first large model based on the corresponding time interval, and its determination criteria include at least one of the following: content information of video frames within the time interval; semantic relevance between content information and metadata or text descriptions; and whether the time interval contains a preset key event type.

[0135] In some embodiments, the preset key event type includes at least one of the following: the preset key event type includes at least one of the following: a preset object is contained in the time interval; a scene transition occurs in the time interval; a preset action or behavior is contained in the time interval; a keyword appears in the audio information corresponding to the time interval; a preset graphical user interface element is contained in the time interval.

[0136] In some embodiments, non-uniform time scaling is implemented using frame repetition or frame interpolation algorithms.

[0137] In some embodiments, non-uniform time scaling processing includes: for adjacent time intervals where the content importance score is higher than a preset importance threshold and the motion complexity is higher than a preset complexity threshold, performing frame interpolation processing between the first and last video frames using a video frame interpolation algorithm based on optical flow; wherein, the motion complexity is determined by calculating the average optical flow vector amplitude between consecutive frames within the adjacent time interval.

[0138] In some embodiments, the video scaling module 202 includes: a total scaling factor determination submodule, configured to set a target total scaling factor M for the target video segment, wherein M is a number greater than 0; a coefficient determination submodule, configured to assign a scaling factor to each time interval based on the content importance score of each time interval, wherein the time interval with a higher content importance score has a larger scaling factor; and a duration processing submodule, configured to determine the processed duration of each time interval based on the scaling factor; wherein the sum of the processed durations of each time interval is equal to the product of the original total duration of the target video segment and the target total scaling factor M.

[0139] In some embodiments, the total multiplier determination submodule is used to: set the target total scaling multiplier based on the original duration of the target video segment.

[0140] In some embodiments, the positioning providing module 204 includes: a button providing submodule for providing a forward jump button and a backward jump button on the video player interface; and a frame jumper module for controlling the video playback to jump forward or backward a predetermined number of frames in response to a trigger operation of either button; wherein the predetermined number of frames are video frames processed by non-uniform time scaling, and the value of the predetermined number is adjusted according to the content importance score distribution of the time interval containing the current playback position and its adjacent time intervals.

[0141] In some embodiments, the predetermined quantity corresponding to the time interval with high content importance score is less than the predetermined quantity corresponding to the time interval with low content importance score.

[0142] In some embodiments, the tag generation module 205 includes: a tag information submodule, used to perform a hash operation on at least one of the timestamp information of the target frame, the unique identification information of the video, and the user identifier to generate tag information.

[0143] In some embodiments, the interactive device for the video progress bar further includes: an upload storage module 206. Figure 2 (Not shown in the image), used to upload the marker information and its associated target frame location information to the server for storage.

[0144] In some embodiments, the interactive device for the video progress bar further includes: a marker acquisition module 207. Figure 2 (not shown in the image), used to retrieve all publicly available tagging information related to the target video segment from the server in response to the target video segment being reloaded and played; tagging classification module 208 ( Figure 2 (Not shown in the image), used to distinguish tag information into a first type of tag created by the current user and a second type of tag created and publicly displayed by other users; Tag display module 209 ( Figure 2 (not shown in the image), used to output a first type of marker and a second type of marker at the corresponding position on the progress bar; wherein the first type of marker and the second type of marker have different visual styles.

[0145] In some embodiments, the interactive device for the video progress bar further includes: a marker triggering module 210 ( Figure 2 (Not shown in the image), used to receive trigger operations on any marker information on the progress bar; Details display module 211 ( Figure 2 (not shown in the image) is used to retrieve and display details related to the tag information in response to a triggering operation. The details include at least one of the following: creator information, comment text, associated keyframe thumbnails, or a shortcut button that jumps to the corresponding location of the tag information.

[0146] In some embodiments, the interactive device for the video progress bar further includes: a marker rendering module 212 ( Figure 2 (not shown in the image) is used to perform differential rendering of at least one of color, texture, or transparency on the portion of the remapped progress bar corresponding to the target video segment, based on the content importance score distribution of the target video segment.

[0147] In some embodiments, the scaling module 203 includes: a ratio calculation submodule, used to calculate the ratio of the total duration of the target video segment after scaling to the original duration of the target video segment; and a stretch display submodule, used to stretch the segment corresponding to the target video segment on the progress bar according to the ratio, while keeping the display ratio of the rest of the progress bar unchanged.

[0148] In some embodiments, the positioning providing module 204 includes: a positioning entry submodule, configured to provide a keyframe preview mode as the entry point for frame-level positioning operations. The positioning entry submodule is configured to: extract frames from the target video segment; input the extracted frames into a second large model and obtain the independent importance score for each frame output by the second large model; based on the independent importance score, filter keyframes from the target video segment to form a keyframe set; display the keyframe set in the form of an image wall in the user interface; and respond to the selection operation of the target keyframe in the image wall, control the video player to jump to the first time point in the target video segment corresponding to the target keyframe.

[0149] In some embodiments, the positioning providing module 204 further includes: a control providing submodule for providing a frame-level positioning control; a fine-tuning positioning submodule for performing frame-level precision forward or backward fine-tuning from a first time point in response to an operation on the frame-level positioning control; and determining the final frame located by the fine-tuning operation; wherein the final frame is the target frame.

[0150] In some embodiments, the location entry submodule is used to: establish a mapping relationship between the independent importance score of each keyframe in the keyframe set and its display size; and determine and display the display size of each keyframe based on the mapping relationship, wherein the keyframe with the higher independent importance score has a larger display size.

[0151] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0152] The interactive device for the video progress bar in this embodiment significantly improves the finesse and controllability of interaction on important segments by using non-uniform time scaling based on content importance and spatial remapping of the progress bar, reducing the operational burden of repeated dragging and trial and error in long videos. Through frame-level positioning and marking, the addressing and revisiting of target moments are more accurate and efficient, forming a semantically enhanced time navigation capability. At the same time, audio-visual consistency and smooth rate control ensure viewability and comprehension continuity, improving the overall browsing efficiency and user experience of long media.

[0153] This disclosure provides a scenario illustration of an interactive method for a video progress bar, as shown in the following embodiments. Figure 3 As shown.

[0154] As previously described, the interactive method for a video progress bar provided in this disclosure is applied to an electronic device. The electronic device is intended to represent various forms of digital computers, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. This electronic device can be installed in or connected to an autonomous vehicle.

[0155] Specifically, the electronic device may perform the following operations: In response to the selection instruction for the target video segment, the target video segment is divided into multiple time intervals, and the content importance score of each time interval is determined. Based on the content importance score of each time interval, the target video segment is subjected to non-uniform time scaling, where the time interval with a higher content importance score is allocated a longer playback duration. The segment in the progress bar corresponding to the target video segment is remapped and displayed according to the time ratio after scaling. Based on the progress bar, frame-level positioning operation is provided. In response to the marking instruction triggered when the positioning of a target frame is completed through frame-level positioning operation, marking information for indicating the position of the target frame is generated.

[0156] It should be understood that Figure 3 The scene diagrams shown are merely illustrative and not restrictive; those skilled in the art can interpret them based on... Figure 3 Even with various obvious changes and / or substitutions to the examples, the resulting technical solutions still fall within the scope of this disclosure.

[0157] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0158] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0159] Figure 4A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0160] like Figure 4 As shown, device 400 includes a computing unit 401, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 402 or a computer program loaded from storage unit 408 into random access memory (RAM) 403. RAM 403 may also store various programs and data required for the operation of device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0161] Multiple components in device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of monitors, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0162] The computing unit 401 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as the interactive method of a video progress bar. For example, in some embodiments, the interactive method of a video progress bar can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the interactive method of a video progress bar described above can be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform interactive methods of the video progress bar by any other suitable means (e.g., by means of firmware).

[0163] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0165] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0167] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0168] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0169] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An interactive method for a video progress bar, comprising: In response to the selection instruction for the target video segment, the target video segment is divided into multiple time intervals, and a content importance score is determined for each time interval; Based on the content importance score of each time interval, the target video segment is subjected to non-uniform time scaling processing, wherein the time interval with the higher content importance score is assigned a longer playback duration. The segment in the progress bar corresponding to the target video clip is remapped and displayed according to the time ratio after scaling. Based on the progress bar, frame-level positioning operations are provided; In response to a marking instruction triggered when the positioning of a target frame is completed through the frame-level positioning operation, marking information indicating the position of the target frame is generated.

2. The method according to claim 1, wherein, The determination of the content importance score for each time interval includes: Obtain the video frames of the target video segment, as well as the metadata or text description related to the target video segment; Input the video frame and its associated metadata or text description into the first main model; Obtain the content importance score for each time interval from the output of the first large model; Wherein, the time interval is a continuous segment of the target video segment on the timeline, which contains one or more frames; The content importance score is determined by the first large model based on the corresponding time interval, and its determination criteria include at least one of the following: Content information of video frames within the time interval; The semantic relevance between the content information and the metadata or text description; Does the time interval contain a preset key event type? 3. The method according to claim 2, wherein, The preset key event types include at least one of the following: The time interval includes preset objects; Scene transitions occur within the time interval; The time interval includes preset actions or behaviors; Keywords appear in the audio information corresponding to the time interval; The time interval contains preset graphical user interface elements.

4. The method according to claim 1, wherein, The non-uniform time scaling process is implemented through frame repetition or frame interpolation algorithms.

5. The method according to claim 4, wherein, The non-uniform time scaling process includes: For adjacent time intervals where the content importance score is higher than the preset importance threshold and the motion complexity is higher than the preset complexity threshold, a video frame interpolation algorithm based on optical flow is used to perform frame interpolation between the first and last video frames. The motion complexity is determined by calculating the average optical flow vector magnitude between consecutive frames within the adjacent time interval.

6. The method according to claim 1, wherein, The non-uniform time scaling process includes: Set the target scaling factor M for the target video segment, where M is a number greater than 0; Based on the content importance score for each time interval, a scaling factor is assigned to each time interval, with the scaling factor being larger for time intervals with higher content importance scores. Based on the scaling factor, the processed duration of each time interval is determined; The sum of the processed durations of each time interval is equal to the product of the original total duration of the target video segment and the total scaling factor M of the target.

7. The method according to claim 6, wherein, Setting the target total scaling factor for the target video segment includes: The total scaling factor of the target video segment is set based on its original duration.

8. The method according to claim 1, wherein, The frame-level positioning operation based on the progress bar includes: The video player interface provides forward and backward navigation buttons; In response to the triggering of any button, control the video playback to jump forward or backward by a predetermined number of frames; The predetermined number of frames are video frames after the non-uniform time scaling process, and the value of the predetermined number is adjusted according to the content importance score distribution of the time interval containing the current playback position and its adjacent time intervals.

9. The method according to claim 8, wherein, The predetermined quantity corresponding to the time interval with high content importance score is less than the predetermined quantity corresponding to the time interval with low content importance score.

10. The method according to claim 1, wherein, The generation of marker information for indicating the location of the target frame includes: The tagging information is generated by performing a hash operation on at least one of the timestamp information of the target frame, the unique identifier information of the video, and the user identifier.

11. The method according to claim 1, wherein, The method further includes: The marker information and its associated target frame location information are uploaded to the server for storage.

12. The method according to claim 11, wherein, The method further includes: In response to the target video segment being reloaded and played, all publicly available tagging information related to the target video segment is obtained from the server; The tagging information is divided into a first type of tag created by the current user and a second type of tag created and made public by other users; At the corresponding positions on the progress bar, the first type of marker and the second type of marker are output; wherein the first type of marker and the second type of marker have different visual styles.

13. The method according to claim 12, wherein, The method further includes: Receive a trigger operation on any of the marker information on the progress bar; In response to the triggering operation, details related to the tag information are obtained and displayed, including at least one of the following: creator information, annotation text, associated keyframe thumbnails, or a shortcut button to jump to the corresponding location of the tag information.

14. The method according to claim 1, wherein, The method further includes: For the portion of the remapped progress bar corresponding to the target video segment, differentiated rendering is performed based on at least one of the following: color, texture, or transparency, according to the content importance score distribution of the target video segment.

15. The method according to claim 1, wherein, The step of remapping and displaying the segment in the progress bar corresponding to the target video clip according to the scaled-up time ratio includes: Calculate the ratio of the total duration of the target video segment after the scaling process to the original duration of the target video segment; On the progress bar, the segment corresponding to the target video clip is stretched and displayed according to the ratio, while keeping the display ratio of the rest of the progress bar unchanged.

16. The method according to claim 1, wherein, The frame-level positioning operation based on the progress bar includes: A keyframe preview mode is provided as the entry point for the frame-level positioning operation; In the keyframe preview mode: Extract frames from the target video segment; The extracted frames are input into the second largest model, and the independent importance score for each frame is obtained from the output of the second largest model. Based on the independent importance score, keyframes are selected from the target video segment to form a keyframe set; The set of keyframes is displayed in the user interface as a picture wall; In response to the selection operation of the target keyframe in the image wall, the video player is controlled to jump to the first time point in the target video segment corresponding to the target keyframe.

17. The method according to claim 16, wherein, After jumping to the first time point, the method further includes: Provide frame-level positioning controls; In response to the operation of the frame-level positioning control, the frame-level precision is finely adjusted forward or backward from the first time point; The final frame located through fine-tuning operations is determined; wherein, the final frame is the target frame.

18. The method according to claim 16, wherein, The process of displaying the keyframe set in the form of an image wall in the user interface includes: Establish a mapping relationship between the independent importance score of each keyframe in the keyframe set and its display size; Based on the mapping relationship, the display size of each keyframe is determined and displayed, wherein the keyframe with the higher independent importance score has a larger display size.

19. An interactive device for a video progress bar, comprising: The scoring determination module is used to divide the target video segment into multiple time intervals in response to the selection instruction for the target video segment, and determine the content importance score for each time interval; The video scaling module is used to perform non-uniform time scaling on the target video segment based on the content importance score of each time interval, wherein the time interval with the higher content importance score is assigned a longer playback duration. The scaling module is used to remap and display the segment in the progress bar that corresponds to the target video clip according to the time scaling ratio. A positioning module is used to provide frame-level positioning operations based on the progress bar; The marker generation module is used to generate marker information indicating the position of the target frame in response to a marker instruction triggered when the positioning of a target frame is completed through the frame-level positioning operation.

20. An electronic device, comprising: At least one processor; as well as A memory that is communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform the method of any one of claims 1-18.

21. A non-transitory computer-readable storage medium storing computer instructions, wherein, Computer instructions are used to cause a computer to perform the method according to any one of claims 1-18.

22. A computer program product comprising a computer program stored on a storage medium, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1-18.

Images