Video playing speed self-adaptive adjusting method

Abstract

The invention discloses a video playing speed adaptive adjustment method, which belongs to the technical field of video processing, and comprises the following steps: reading video data to be processed, preprocessing the video data, generating a video frame and a corresponding video frame index, and storing the generated video frame into a video frame buffer area; selecting at least one group of video frames from the video frame buffer area, detecting the extracted video frames through a self-adaptive detection algorithm, and outputting an analysis result which comprises starting time of the video frames and a picture change value; and according to an analysis result, setting a playing speed self-adaptive adjustment strategy, and carrying out self-adaptive adjustment on the playing speed of video playing. In the implementation process of the technical scheme provided by the invention, the real-time video picture change is detected during playing, the video picture change value in the video is analyzed through the detection algorithm, and the playing speed and the playing position are adaptively and dynamically adjusted, so that the picture change is reduced, and the playback efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method for adaptively adjusting video playback speed. Background Art

[0002] With the development of the Internet and mobile communications, video content is becoming richer and users' requirements for playback experience are increasing. For example, in video conferencing, online classrooms, video surveillance and other systems, in order to save and review, a recording function is generally provided so that users can watch back. During the playback process, the playback speed can be adjusted to quickly locate the video and save playback time.

[0003] However, there are still some problems that affect the experience in the existing video playback technology. For example, the traditional video fast-forwarding method is achieved by using a higher frame rate or skipping a certain number of video frames. This solution will cause drastic changes in the picture during the fast-forward process, which is easy to cause visual fatigue, and the corresponding audio will also change drastically, affecting the recognition of video content. At the same time, the current fast-forward multiples are usually fixed, such as 1x, 1.5x, 2x, etc., and there is no adaptive adjustment of the fast-forward multiple according to the video content. The user experience is poor, and optimization of these problems is urgent.

[0004] Therefore, it is necessary to provide a method for adaptively adjusting video playback speed to solve the above problems.

[0005] It should be noted that the above information disclosed in this background technology section is only for understanding the background technology of the present application concept, and therefore, it may contain information that does not constitute the prior art. Summary of the invention

[0006] Based on the above problems existing in the prior art, the problem to be solved by the present application is: to provide a method for adaptively adjusting the video playback speed, so as to achieve the effect of adjusting the playback speed by combining the degree of video change.

[0007] The technical solution adopted by the present application to solve the technical problem is: a method for adaptively adjusting video playback speed, the method for adaptively adjusting video playback speed comprising:

[0008] Read the video data to be processed, pre-process the video data, generate video frames and corresponding video frame indexes, and store the generated video frames in a video frame buffer;

[0009] Selecting at least one group of video frames from the video frame buffer, and detecting the extracted video frames by using an adaptive detection algorithm, and outputting analysis results, the analysis results including the start time of the video frames and the picture change value;

[0010] According to the analysis results, a playback speed adaptive adjustment strategy is set to adaptively adjust the playback speed of the video.

[0011] During the implementation of the technical solution of the present application, by detecting real-time video picture changes during playback, analyzing the video picture change values ​​in the recording through a detection algorithm, and adaptively and dynamically adjusting the playback speed and playback position, picture changes are reduced and playback efficiency is improved.

[0012] Furthermore, preprocessing the video data specifically includes the following steps:

[0013] Determine the format and encoding of video data and perform decoding processing;

[0014] Extract video key frames and establish the temporal relationship between key frames;

[0015] Generate video frame indexes based on the temporal relationship to facilitate quick retrieval and analysis of the degree of change of each video frame, thereby determining the fast-forward multiple of video playback.

[0016] Furthermore, the start time of the video frame is obtained by associating the video frame index, and the picture change value reflects the difference between different video frames.

[0017] Further, detecting the extracted video frame by an adaptive detection algorithm includes the following steps:

[0018] Perform feature extraction on the extracted video frames, the feature extraction including pixel gray value, edge feature, texture feature, motion vector;

[0019] Establish a Kalman filter model, and use the pixel coordinates and color values ​​as state vector elements of the model;

[0020] Set the state transfer matrix, obtain the picture noise covariance, and remove background noise interference from the video frame;

[0021] The Kalman filter model is used to predict the video image features at the current moment, and the estimated value of the video image features is updated, and the estimated value of the feature at the current moment is compared with the estimated value of the feature at the previous moment to generate a feature difference;

[0022] Set a feature difference threshold and compare the generated feature difference with the feature difference threshold to determine whether the video frame has changed.

[0023] Furthermore, for video frames whose feature difference does not exceed the threshold, compression processing is performed to reduce the data transmission burden, and the feature difference threshold is dynamically adjusted according to the video analysis requirements in different scenarios.

[0024] Furthermore, the adaptive adjustment strategy for the playback speed includes: in a scene with drastic motion changes, smoothly transitioning the playback speed, and in a static scene with minor changes, directly skipping a section of the picture.

[0025] Furthermore, the adaptive adjustment strategy is implemented using a threshold correspondence method, which includes presetting multiple thresholds, each threshold corresponding to a different playback speed adjustment scheme. If the video frame picture change is higher than the preset threshold, the playback speed is increased to the speed level corresponding to the threshold. If the video frame picture change is lower than the preset threshold, the playback position is jumped to after this group of video frames.

[0026] A video playback speed adaptive adjustment system, the video playback speed adaptive adjustment system comprising:

[0027] A preprocessing module is used to read the video data to be processed, preprocess the video data, generate video frames and corresponding video frame indexes, and store the generated video frames in a video frame buffer;

[0028] An adaptive detection module, used to select at least one group of video frames from the video frame buffer, detect the extracted video frames through an adaptive detection algorithm, and output analysis results, which include the start time of the video frames and the picture change value;

[0029] The adaptive adjustment module is used to set a playback speed adaptive adjustment strategy according to the analysis results, and adaptively adjust the playback speed of the video.

[0030] The beneficial effect of the present application is that the present application provides a method for adaptively adjusting the video playback speed, which detects real-time video picture changes during playback, analyzes the video picture change values ​​in the recording through a detection algorithm, and adaptively and dynamically adjusts the playback speed and playback position, thereby reducing picture changes, improving playback efficiency, and reducing the consumption of computing resources, while ensuring the complete presentation of key pictures.

[0031] In addition to the above-described purposes, features and advantages, the present application also has other purposes, features and advantages. The present application will be further described in detail with reference to the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0033] In the attached picture:

[0034] Figure 1 This is a schematic diagram of the overall process of a method for adaptively adjusting video playback speed in this application;

[0035] Figure 2 This is a schematic diagram of the module structure of a video playback speed adaptive adjustment system in this application. DETAILED DESCRIPTION

[0036] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0038] Embodiment 1: Figure 1 As shown, the present application provides a method for adaptively adjusting video playback speed, which is applied to the video playback process and can also be applied to a video player. By detecting real-time video picture changes and analyzing the degree of video picture changes, the video fast-forward multiple is adaptively adjusted. The method includes the following steps:

[0039] Step 10: Read the video data to be processed, pre-process the video data, generate video frames and corresponding video frame indexes, and store the generated video frames in a video frame buffer;

[0040] In the application process of the embodiment of the present application, in order to reduce a large number of invalid copies in the processing process, it is necessary to pre-process the video data to be processed, and generate multiple video frames, and store the generated multiple video frames in a pre-allocated video frame buffer. In the subsequent call and analysis process, it can be directly called from the video frame buffer without searching and copying the original video data, thereby improving the analysis efficiency. At the same time, a video frame index is also generated in the pre-processing process, which facilitates the rapid positioning of each frame and provides data support for the subsequent adaptive adjustment of the playback speed.

[0041] The preprocessing of the video data specifically includes the following steps:

[0042] Step 101: Determine the format and encoding of the video data and perform decoding processing; in different systems, the video data may have different formats and encoding forms, so during preprocessing, it is first necessary to determine the format and encoding of the video data and decode it to ensure that the video data can be correctly parsed and processed, wherein the format and encoding of the video data can be obtained by searching the metadata information of the video stream, and the decoding processing is performed based on the metadata information. For details, please refer to the prior art and will not be described in detail in this embodiment.

[0043] Step 102: extract video key frames and establish a time sequence relationship between key frames; extract key frames by analyzing the video content, identifying representative images, and establishing a time sequence relationship between key frames to form the skeleton of the video content. This step helps to quickly locate important images during video playback and provides a basis for adaptive adjustment. After the time sequence relationship between key frames is established, the computational complexity during video playback can be effectively reduced, making subsequent adaptive adjustment more accurate and efficient.

[0044] Step 103: Generate a video frame index based on the time sequence relationship to facilitate rapid retrieval and analysis of the degree of change of each video frame, thereby determining the fast-forward multiple of the video playback.

[0045] When generating video frames, it is also necessary to generate video frame indexes based on the timing relationship, so as to quickly match key frames during the adaptive adjustment process, realize real-time adjustment of the video playback speed, and facilitate the rapid positioning of video frames in the subsequent video analysis process, thereby optimizing the data processing process.

[0046] Step 20: selecting at least one group of video frames from the video frame buffer, and detecting the extracted video frames by using an adaptive detection algorithm, and outputting analysis results, the analysis results including the start time of the video frames and the picture change value;

[0047] After the video data to be processed is sent to the video frame buffer, when analyzing, at least one group of video frames is selected from the video frame buffer, the start and end of the selected video frames are both key frames, and the extracted video frames are detected by applying an adaptive detection algorithm, the degree of picture change thereof is analyzed, and the start time and picture change value of the video frame are output, wherein the start time of the video frame is obtained by associating the video frame index, and the picture change value reflects the difference between the video frames, providing a reference for adaptively adjusting the playback speed;

[0048] Specifically, detecting the extracted video frame by using an adaptive detection algorithm includes the following steps:

[0049] Step 201: extracting features from the extracted video frame, the features including pixel grayscale value, edge features, texture features, and motion vectors;

[0050] Traditional video picture change detection methods are mainly based on pixel comparison, frame difference method, etc., but these methods have problems such as large amount of calculation, sensitivity to illumination changes and noise, etc., and are not suitable for this embodiment. Therefore, a feature extraction method is adopted, and the extracted features will be used to more carefully analyze the differences between video frames to more accurately determine the degree of change of video frames;

[0051] Step 202: Establish a Kalman filter model, and combine pixel coordinates and color values ​​as state vector elements of the model;

[0052] The Kalman filter model is a model that can realize state estimation and prediction. It has the advantages of simple calculation and certain robustness to noise. It combines pixel coordinates and color values ​​as state vector elements, which can effectively track the movement trajectory and color changes of objects in video frames, and provide a dynamic reference for adaptive adjustment. Through the prediction and update of the Kalman filter model, it can correct small errors between video frames in real time to ensure the accuracy of the analysis results. At the same time, this process also provides important data support for subsequent video compression and optimized playback experience.

[0053] Step 203: setting a state transfer matrix, obtaining the picture noise covariance, and removing background noise interference from the video frame;

[0054] In the process of analyzing and processing video frames, there will be a lot of background noise interference, which will not only affect the accuracy of the analysis results, but also affect the judgment of the degree of change of the video frames. Therefore, by obtaining the picture noise covariance, and then combining it with the state transfer matrix to obtain the state transfer prediction formula, the background noise interference can be eliminated;

[0055] Step 204: predicting the video picture features at the current moment through the Kalman filter model, and updating the estimated value of the video picture features, comparing the estimated value of the features at the current moment with the estimated value of the features at the previous moment, and generating a feature difference;

[0056] The Kalman filter model can predict the video picture features at the current moment. When a new video frame is obtained, the features of the current frame are extracted, and the estimated values ​​of the video picture features are updated according to the update steps of the Kalman filter to generate feature differences. This difference will directly reflect the degree of change between video frames.

[0057] Step 205: setting a feature difference threshold, and comparing the generated feature difference with the feature difference threshold to determine whether the video frame has changed;

[0058] By setting the threshold, the degree of change of the video frame can be effectively quantified. For frames whose feature difference exceeds the threshold, it is determined that there is a significant change in the video frame, and subsequent software and hardware coordination optimization is performed. For video frames whose feature difference does not exceed the threshold, it is considered that the video frame has not changed much and can be compressed to reduce the data transmission burden. In addition, combined with the accuracy of the Kalman filter model prediction, the feature difference threshold can be dynamically adjusted to meet the video analysis needs in different scenarios.

[0059] Step 30: According to the analysis results, a playback speed adaptive adjustment strategy is set to adaptively adjust the playback speed of the video playback;

[0060] After analyzing the extracted video frames, according to the generated analysis results, it is necessary to set an adaptive playback speed adjustment strategy to optimize the viewing experience. This strategy will combine the dynamic data provided by the Kalman filter model to make intelligent adjustments to the degree of video frame changes, ensuring that the playback speed can smoothly transition in scenes with drastic motion changes to avoid frame skipping or stuttering. In static scenes with small changes, appropriately increasing the playback speed or skipping a section of the screen can not only improve viewing efficiency, but also effectively reduce data transmission volume, taking into account both fluency and economy, and achieving a comprehensive improvement in user viewing experience.

[0061] Through intelligent control of the playback speed, the overall playback efficiency is improved while ensuring the complete presentation of the video content. In addition, combined with user behavior analysis, the adaptive adjustment strategy can be further optimized to ensure that the adjustment of the video playback speed can better meet the needs of different user groups, improve user satisfaction and viewing stickiness. At the same time, this strategy also helps to reduce server load and improve the efficiency and quality of video processing;

[0062] Specifically, the adaptive adjustment strategy is implemented using a threshold correspondence method, which includes presetting multiple thresholds, each threshold corresponding to a different playback speed adjustment scheme. If the video frame picture change is higher than the preset threshold, the playback speed is increased to the speed level corresponding to the threshold. If the video frame picture change is lower than the preset threshold, the playback position is jumped to after this group of video frames, thereby saving viewing time and reducing data transmission pressure. This strategy is particularly suitable for video content with rapidly changing scenes such as sports events and action movies, while in dialogue or static scenes, accelerated playback can be effectively used. In addition, the flexible application of this strategy can optimize the utilization of network bandwidth while ensuring video fluency and achieve efficient resource allocation.

[0063] Embodiment 2: Figure 2 As shown, the present application also proposes a video playback speed adaptive adjustment system, which runs the adaptive adjustment method in embodiment 1, and the system includes:

[0064] A preprocessing module is used to read the video data to be processed, preprocess the video data, generate video frames and corresponding video frame indexes, and store the generated video frames in a video frame buffer;

[0065] An adaptive detection module, used to select at least one group of video frames from the video frame buffer, detect the extracted video frames through an adaptive detection algorithm, and output analysis results, which include the start time of the video frames and the picture change value;

[0066] The adaptive adjustment module is used to set a playback speed adaptive adjustment strategy according to the analysis results, and adaptively adjust the playback speed of the video.

[0067] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for adaptively adjusting video playback speed, characterized in that: The video playback speed adaptive adjustment method includes: Read the video data to be processed, pre-process the video data, generate video frames and corresponding video frame indexes, and store the generated video frames in a video frame buffer; Selecting at least one group of video frames from the video frame buffer, and detecting the extracted video frames by using an adaptive detection algorithm, and outputting analysis results, the analysis results including the start time of the video frames and the picture change value; According to the analysis results, a playback speed adaptive adjustment strategy is set to adaptively adjust the playback speed of the video.

2. The method for adaptively adjusting video playback speed according to claim 1, characterized in that: The preprocessing of video data specifically includes the following steps: Determine the format and encoding of video data and perform decoding processing; Extract video key frames and establish the temporal relationship between key frames; Generate video frame indexes based on the temporal relationship to facilitate quick retrieval and analysis of the degree of change of each video frame, thereby determining the fast-forward multiple of video playback.

3. The method for adaptively adjusting video playback speed according to claim 1, characterized in that: The start time of the video frame is obtained by associating the video frame index, and the picture change value reflects the difference between different video frames.

4. The method for adaptively adjusting video playback speed according to claim 1, characterized in that: Detecting the extracted video frames using an adaptive detection algorithm includes the following steps: Perform feature extraction on the extracted video frames, the feature extraction including pixel gray value, edge feature, texture feature, motion vector; Establish a Kalman filter model, and use the pixel coordinates and color values ​​as state vector elements of the model; Set the state transfer matrix, obtain the picture noise covariance, and remove background noise interference from the video frame; The Kalman filter model is used to predict the video image features at the current moment, and the estimated value of the video image features is updated, and the estimated value of the feature at the current moment is compared with the estimated value of the feature at the previous moment to generate a feature difference; Set a feature difference threshold and compare the generated feature difference with the feature difference threshold to determine whether the video frame has changed.

5. The method for adaptively adjusting video playback speed according to claim 4, characterized in that: For video frames whose feature difference does not exceed the threshold, compression processing is performed to reduce the data transmission burden, and the feature difference threshold is dynamically adjusted according to the video analysis requirements in different scenarios.

6. The method for adaptively adjusting video playback speed according to claim 1, characterized in that: The play speed adaptive adjustment strategy includes: in a scene with drastic motion changes, smoothly transitioning the play speed, and in a static scene with small changes, directly skipping a section of the picture.

7. The method for adaptively adjusting video playback speed according to claim 1, characterized in that: The adaptive adjustment strategy is implemented using a threshold correspondence method, which includes presetting multiple thresholds, each threshold corresponding to a different playback speed adjustment scheme. If the video frame picture change is higher than the preset threshold, the playback speed is increased to the speed level corresponding to the threshold. If the video frame picture change is lower than the preset threshold, the playback position is jumped to after this group of video frames.

8. A video playback speed adaptive adjustment system, characterized by: The video playback speed adaptive adjustment system includes: A preprocessing module is used to read the video data to be processed, preprocess the video data, generate video frames and corresponding video frame indexes, and store the generated video frames in a video frame buffer; An adaptive detection module, used to select at least one group of video frames from the video frame buffer, detect the extracted video frames through an adaptive detection algorithm, and output analysis results, which include the start time of the video frames and the picture change value; The adaptive adjustment module is used to set a playback speed adaptive adjustment strategy according to the analysis results, and adaptively adjust the playback speed of the video.

9. The video playback speed adaptive adjustment system according to claim 8, characterized in that: Used to implement the video playback speed adaptive adjustment method as described in any one of claims 1 to 7.

Images